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Introduction 


The  Military  Interoperable  Digital  Hospital  Testbed  (MIDHT)  is  a  six-year  program  of  research  to 
develop  a  real-world  testbed  environment  in  Southwestern  Pennsylvania.  The  purpose  is  to 
research  and  evaluate  Health  Information  Exchange  (HIE)  and  health  information  technology 
(HIT)  and  services  (HITS)  that  make  health  information  readily  available  to  consumers  and 
providers.  Ideally  this  will  allow  for  the  secure  transfer  of  information  between  private  sector 
rural  providers,  federal  partners  and  patients.  MIDHT  has  defined  requirements  and  solutions 
to  optimize  healthcare  resources  for  rural  communities  and  identified  lessons  learned  and  best 
practices  that  benefit  both  the  global  Military  Health  System  (MHS)  environment  and 
stakeholders  in  the  local  region.  The  Department  of  Defense  (DoD)  and  Conemaugh  Memorial 
Medical  Center  (CMMC)  have  common  requirements  for  HIE,  connecting  disparate  systems  and 
providers  and  enabling  secure  provider-provider  and  provider-consumer  e-communications. 
Minimal  evidence  is  available  on  what  business,  clinical  and  technical  solutions  can  be  used  to 
overcome  the  lack  of  specialists,  infrastructure  and  geographical  barriers  associated  with  the 
delivery  of  care  in  rural  communities. 

Arm  1.  The  Impact  of  Medication  Dispensing/Administration  Technology  Within  a  Rural 
Healthcare  System. 

In  order  to  improve  efficiency  and  safety  of  medication  dispensing  and  administration  within  an 
inpatient  hospital  setting,  a  complementary  set  of  health  information  technologies  have  been 
implemented.  A  centrally  managed  pharmacy  robotics  system  implemented  in  2011  works  in 
conjunction  with  bar  coded  medications  administered  at  the  bed  side  on  many  units  at  CMMC 
and  Meyersdale  Medical  Center  (MYMC).  Research  activities  have  focused  on  medication 
errors,  provider  workflow,  provider  satisfaction  and  financial  data. 

Arm  2.  Health  Information  Exchange  (HIE)  via  the  Nationwide  Health  Information  Network 
(NwHIN). 

Building  upon  work  completed  the  previous  year,  CMMC  continued  efforts  on  health 
information  exchange  using  the  eHealth  Exchange  (formerly  NwHIN)  standards  and 
specifications.  CMMC  has  been  in  production  with  the  James  E.  Van  Zandt  VA  Medical  Center 
in  Altoona,  PA  for  one  year  as  part  of  the  14th  Virtual  Lifetime  Electronic  Record  (VLER)  pilot  in 
the  United  States.  CMMC  has  made  tremendous  strides  in  a  number  of  activities,  including  a 
CONNECT  gateway  upgrade,  testing  certification,  training  additional  providers  and  increasing 
the  number  of  correlated  patients. 

CMMC  received  a  formal  contract  modification  on  August  26,  2013.  The  modification  extended 
the  contract  at  no-cost  for  one  year  through  October  27,  2014  and  also  included  a  revised 
budget  (dated  May  2,  2013).  CMMC's  subcontract  with  Northrop  Grumman  will  expire  on 
September  30,  2013. 
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Body 


Arm  1:  The  Impact  of  Medication  Dispensing/Administration  Technology  Within  a  Rural 
Healthcare  System 

Subtask  1.1  Implement  pharmacy  robotic  technology  and  bar-coded  enabled  medication 
administration  (BCMA)  in  an  acute  hospital  system  setting. 

Pharmacy  Robotics  /  Bar  Code  Medication  Administration  (BCMA) 

MedCarousel2  Implementation  -  CMMC 

CMMC  finalized  the  second  MedCarousel  dispensing  system  placement  in  the  centralized 
pharmacy  (see  diagram  in  Appendix  1).  CMMC  worked  with  an  outside  engineering  firm  to 
prepare  documents  that  were  submitted  to  the  PA  Department  of  Health  in  January  2013  for 
approval  before  physical  accommodation  work  began.  The  amount  of  site  work  required  was 
much  less  than  with  first  MedCarousel  due  to  the  fact  that  the  second  unit  was  not  as  tall  as  the 
first  one  and  would  not  protrude  through  the  ceiling.  Floor  support  was  the  main  required 
change. 

The  second  MedCarousel  implementation  was  implemented  in  the  pharmacy  on  April  8,  2013 
(Figure  1.  MedCarousel  2.).  CMMC  streamlined  the  manual  pick  processing  that  occurs  when  a 
medication  is  not  able  to  be  stocked  in  the  robot.  This  also  has  helped  reduce  the  number  of 
open  bulk  medication  bottles  and  decrease  inventory  slightly  because  of  increased  efficiency. 
The  second  MedCarousel  has  also  helped  with  the  picking  processes  since  of  the  oral 
medications  are  located  in  the  MedCarousel.  The  barcode  system  is  fully  engaged  and  ensures 
patient  safety  each  time  a  medication  is  removed  from  the  MedCarousel.  Technician  time  is 
better  spent  doing  other  tasks  than  searching  for  a  missing  medication.  The  second 
MedCarousel  has  helped  CMMC  achieve  a  better  work  flow  pattern  in  the  drug  distribution 
area. 

The  additional  MedCarousel  has  also  helped  with  the  ordering  and  restocking  of  inventory 
when  it  arrives  within  the  department  from  the  manufacturer.  With  barcode  scanning,  CMMC 
can  ensure  that  the  medication  is  in  its  proper  place  when  being  restocked.  As  with  the  first 
MedCarousel,  PAR  levels  are  setup  to  help  keep  inventory  levels  sufficient  to  meet  patient 
needs. 
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Figure  1.  MedCarousel  2  in  pharmacy. 

McKesson  has  installed  the  BCMA  Analytics  feed  into  Conemaugh's  performance  management 
system  called  HBI.  The  Decision  Support  department  conducted  user  training  for  nurse 
managers  and  directors  on  the  use  of  HBI  to  interact  directly  with  the  BCMA  data.  This  is  a  very 
rich  analytics  tool  and  once  trained,  nurse  leaders  are  able  to  drill  into  the  data  to  identify 
specific  staff,  nurse  stations,  and  medications  that  require  follow-up  and  possible  re-training. 
Screenshots  located  in  Appendix  2  provide  a  look  and  feel  of  the  BCMA  highlights  via  HBI  and 
the  different  customizable  view  levels  (by  unit,  by  med,  by  staff,  etc.). 

Medication  Delivery  Process  -  CMMC 

Under  the  current  process,  paper  envelopes  are  delivered  to  nursing  units  each  day.  Nursing 
then  must  transfer  the  medications  from  the  envelopes  into  the  plastic  bins  housed  in  the 
computerized  med  carts.  Unused  medications  are  returned  back  to  the  pharmacy  in  the  same 
envelopes.  This  process  has  created  extra  steps  for  nursing  and  thus  has  been  identified  as  an 
area  of  dissatisfaction.  Additionally,  since  the  envelopes  are  re-used  as  a  cost  saving  strategy, 
the  extra  handling  associated  with  this  process  leads  to  the  envelopes  becoming  damaged.  This, 
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in  turn,  contributes  to  dispensing  malfunctions  from  the  robot.  The  Pharmacy  department  has 
since  worked  closely  with  nursing  to  pilot  a  new  "exchange  bin"  method  of  delivering 
medications  to  inpatient  units.  In  this  workflow,  nursing  gets  patient-identified  plastic  bins  with 
the  robot-filled  medications  from  the  current  day.  These  bins  are  exchanged  with  bins  from  the 
preceding  day  for  the  same  patient  in  the  med  cart.  The  expected  advantages  are:  1.  Envelopes 
never  leave  the  pharmacy  so  they  are  much  less  likely  to  get  mis-handled  and  damaged  (saves 
cost)  2.  Nursing  does  not  have  to  transfer  meds  from  envelope  to  bin  and  then  transfer  unused 
meds  from  bin  to  envelope  (less  chance  for  error).  Feedback  from  the  first  two  pilot  units  has 
been  positive.  During  the  next  quarter.  Pharmacy  anticipates  to  recommend  full  adoption  of 
this  process  to  nursing  leadership.  We  are  still  working  with  Rubbermaid  to  finalize  the  pricing 
quote  and  purchase  order. 

Overall  BCMA  Deployment  -  CMMC 

Table  1  demonstrates  the  number  of  medication  administration  attempts,  medications  given 
and  the  total  administrations  using  barcode  scanning  during  the  last  three  months.  BCMA 
utilization  continues  to  run  at  95%.  As  seen  in  Table  1,  the  difference  between  medication 
administration  attempts  and  the  actual  medication  given  may  be  due  to  the  following  reasons, 
including  patient  refusal  or  a  vital  sign/laboratory  parameter  exceeding  protocol  administration 
range. 


Month 

Medication  was 

Charted 

Total  Med 

Admin 

Attempts 

Given 

%  Given 

Barcode 

Scanned 

Count 

%  Barcode 

Scanned 

Apr-13 

160,314 

134,946 

84.2% 

127,571 

94.5% 

May-13 

177,850 

150,901 

84.8% 

142,829 

94.7% 

Jun-13 

120,492 

102,227 

84.8% 

97,222 

95.1% 

Total 

458,656 

388,074 

84.6% 

367,622 

94.7% 

Tablet.  BCMA  Adoption. 


Miners  BCMA  Deployment 

In  May,  Miners  Medical  Center's  (MIMC)  pharmacy  medication  master  file  was  successfully 
synchronized  with  that  of  Memorial/Meyersdale's.  A  single  medication  master  file  now  serves 
all  three  Conemaugh  hospitals.  This  was  a  multi-month  effort  that  completed  a  key 
prerequisite  to  implementing  BCMA  at  Miners.  As  this  task  was  completed,  the  MIS  Clinical 
Analysts  began  the  order  build  process.  Meetings  were  held  at  the  Miners  facility  with  nursing 
and  ancillary  staff  to  design  the  workflow  process.  The  system  build  is  underway  (approximately 
50%  complete).  Upon  completion  of  the  build,  a  thorough  system  test  will  take  place.  Planning 
and  implementation  of  user  training  is  scheduled  to  occur  in  the  next  quarter.  MIS  worked  with 
MiMC  Nursing  Administration  to  identify  super  users  to  facilitate  BCMA  training  and  on-site 
support,  similar  to  the  process  used  for  the  Memorial  implementation.  MIS  recruited  the 
services  of  several  nursing  super  users  to  dedicate  time  to  these  tasks.  BCMA  deployment  at 
Miners  has  been  delayed  due  to  implementation  of  the  McKesson  ER12  upgrade  and  is 
scheduled  for  the  first  quarter  of  calendar  year  2014. 
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School  of  Nursing  BCMA  Deployment 

Conemaugh  staff  finalized  the  School  of  Nursing  device  needs  for  student  nurses  in  November 
2012.  Additional  Rubbermaid  medication  carts  were  deployed  in  December  2012  after  receipt 
from  vendor.  The  primary  purpose  was  to  improve  student  exposure  to  the  current  BCMA 
process  and  enhance  the  learning  experience. 

Subtask  1.2  Research  and  analyze  the  resulting  technological  impact  on  medication  errors, 
pharmacist  productivity,  nurse  satisfaction/workflow  and  patient  satisfaction. 

Nursing  Surveys  at  CMMC  &  MYMC 

A  total  of  25  CMMC  nurses  completed  the  survey  during  the  reporting  period.  The  survey  was 
closed  on  October  5,  2012.  Final  statistical  analysis  has  been  completed  for  the  CMMC  dataset. 
The  addition  of  25  surveys  did  not  change  the  results  and  conclusions  of  the  non-pairwise 
analysis  (one-way  ANOVA)  reported  in  the  2012  Annual  Report. 

The  pairwise  (same  person  completing  survey  at  multiple  time  points)  analysis  is  as  follows: 

A  repeated-measures  analysis  of  variance  (RM-ANOVA)  was  completed  between  Baseline  and 
Post  1  and  Baseline  and  Post  2  periods  over  the  same  questions  as  were  analyzed  for  the  one¬ 
way  ANOVA.  The  Sidak  formula  was  used  to  adjust  for  multiple  comparisons.  Overall  alpha 
(family-wise)  remained  at  0.05.  These  results  support  the  conclusions  of  non-pairwise  analysis. 
Unfortunately  respondent  error  with  creation  of  the  "SUM"  ID  and  a  lack  of  trust  by  nurses  in 
order  to  protect  their  identity  prevented  a  higher  sample  size. 


Time  Periods 

Question 

Later  time 

-  baseline 

Mean 

Difference 

over  TIME 

p-value 

Correction  for  non¬ 
sphericity 

Baseline  and  Post  1, 

N  =  35 

Q30 

decrease 

-1.143 

0.008 

Q04 

NS 

Q05 

NS 

Q06 

NS 

Q07 

decrease 

-0.788 

0.011 

Univariate  results, 
Greenhouse-Geisser 

Q13 

NS 

Q15 

NS 

Q18 

NS 

Baseline  and  Post  2, 

N  =  26 

Q30 

NS 

Q04 

increase 

0.846 

0.023 

Univariate  results, 
Greenhouse-Geisser 

Q05 

NS 
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Q06 

NS 

Q07 

decrease 

-0.808 

0.034 

Univariate  results, 
Greenhouse-Geisser 

Q13 

NS 

Q15 

increase 

0.192 

0.019 

Univariate  results, 
Greenhouse-Geisser 

Q18 

NS 

NS  =  No  Significance 


Table  2.  Repeated-  Measures  ANOVA. 

Q30:  Overall,  how  satisfied  are  you  with  the  current  medication  administration  system? 
Answer  Choices:  1  =  Completely  Dissatisfied  <>  5  =  neither  <>  10  =  Completely  Satisfied 

♦  means.  Baseline  =  5.63;  Post  1  =  4.49 

♦  The  results  above  indicate  a  statistically  significant  decrease  in  satisfaction  of  BCMA 
from  baseline  to  post  1. 


A  6-point  Likert  scale  was  used  for  the  remaining  questions  under  analysis: 

1  Strongly  Agree  3  Slightly  Agree  5  Moderately  Disagree 

2  Moderately  Agree  4  Slightly  Disagree  6  Strongly  Disagree 

0  NA  Not  Applicable 

Please  refer  to  the  tables  below  for  quantification  of  the  mean  score  for  the  following  questions 
by  time  period. 

Q04:  Because  of  information  available  through  the  current  medication  administration  system  I 
know  both  the  intended  actions  and  side  effects  of  medications  I  administer. 

♦  The  results  above  indicate  a  statistically  significant  improvement  of  perception  of 
knowledge  after  implementation  of  BCMA  as  compared  to  baseline. 

Q07:  The  current  medication  administration  system  provides  me  with  information  to  know  that 
a  medication  order  has  been  checked  by  a  pharmacist  before  I  administer  the  medication. 

♦  The  results  above  indicate  a  statistically  significant  decrease  in  the  perception  of  the 
knowledge  that  the  medication  order  has  been  checked  by  a  pharmacist  before 
administration  by  the  nurse  after  implementation  of  BCMA  as  compared  to  baseline. 

Q15:  The  current  medication  administration  system  is  effective  in  reducing  and  preventing 
medication  errors. 
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♦  The  results  above  indicate  a  statistically  significant  improvement  of  perception  of  the 
effectiveness  of  BCMA  to  reduce  and  prevent  medication  errors  as  compared  to 
baseline. 


Descriptive  Statistics:  Baseline(  1)  -  Post  1  (  2) 


Mean 

Std.  Deviation 

N 

Q07  1 

2.94 

1.694 

33 

Q07  2 

2.15 

.906 

33 

Table  3.  Descriptive  Statistics. 


Descriptive  Statistics:  Baseline(  1)-Post2(  3) 


Mean 

Std.  Deviation 

N 

Q04  1 

2.73 

1.589 

26 

Q04  3 

3.58 

1.901 

26 

Q07  1 

3.31 

1.436 

26 

Q07  3 

2.50 

1.273 

26 

Q15  1 

3.65 

1.231 

26 

Q15  3 

3.00 

1.497 

26 

The  second  POST  survey  was  distributed  to  nursing  staff  at  MYMC  in  early  January  2013.  A  total 
of  17  surveys  were  completed  within  four  weeks,  with  two  of  those  surveys  not  100%  complete 
so  the  sample  size  varies  from  15-17  depending  on  question. 

Below  are  the  final  results  for  all  three  survey  distributions  at  Meyersdale: 


Responses  to  Questions  4-19  and  23-29  were  coded  0-6,  inclusive,  with  the  following 
meaning: _ 


0  =  N/A  (not  applicable) 

1  =  Strongly  Agree 

4  =  Slightly  Disagree 

2  =  Moderately  Agree 

5  =  Moderately  Disagree 

3  =  Slightly  Agree 

6  =  Strongly  Disagree 

Q30  was  coded  from  1-10  with  a  between  response  interval  =  1,  where: 

1  =  completely  dissatisfied,  5  =  neither  satisfied/dissatisfied,  10  =  completely  satisfied 

These  response  scales  are  ordinal  and  approximately  interval.  In  order  to  take  advantage  of  the 
more  powerful  parametric  tests,  the  scale  has  been  assumed  continuous.  As  this  is  an 
assumption  for  analysis  only,  non-parametric  tests  were  also  applied  to  check  consistency  of 
the  analysis. 
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The  following  analyses  are  between-group,  pre  vs.  post  1  vs.  post  2  study  time  periods,  with  a 
family-wise  alpha  criteria  =  0.05.  When  homogeneity  of  variances  held,  the  Bonferroni 
correction  was  applied  for  pairwise  comparisons;  when  not,  Games-Howell  was  used. 

Survey  Section  3:  Q04-08 

Homogeneity  of  variances  (over  study  time  periods)  held  for  all  questions. 

Only  Q04  showed  a  statistically  significant  omnibus  ANOVA,  p-value  =  0.024.  The  pairwise 
results  follow  and  show  significant  results  between  the  pre  and  post  1  periods,  p-value  =  0.037. 


Multiple  Comparisons 


Bonferroni 


Dependent  Variable  (1)  Study  Time  Period  (J)  Study  Time  Period 

Mean 

Difference 

<W> 

Std. 

Error 

s/a. 

95%  Confidence  Interval  t 

Lower  Bound 

Upper  Bound 

S3  Q04:  Because  of 
information  available 
through  the  current 
medication 

administration  system  1 
know  both  the  intended 
actions  and  side  effects 
of  medications  1 
administer 

PRE  (before  BCMA) 
N.PRE  -  19 

Post  1  (BCMA  stabilization) 

-1  498' 

.579 

.037 

-2.93 

-.07 

Post  2  (BCMA) 

-1.480 

641 

.073 

-3.06 

.10 

Post  1  (BCMA  stabilization) 
N,Post  1  =  26 

PRE  (before  BCMA) 

1.498* 

.579 

.037 

.07 

2.93 

Post  2  (BCMA) 

.018 

.599 

1.000 

-1.46 

1.49 

Post  2  (BCMA) 

N, Post  2  =  17 

PRE  (before  BCMA) 

1.480 

641 

.073 

-10 

3.06 

Post  1  (BCMA  stabilization) 

-.018 

599 

1.000 

-1.49 

1.46 

*  The  mean  difference  is  significant  at  the  0-05  level. 


Table  4.  Question  4  Statistical  Results. 


Figure  2.  Graphical  Representation  of  Question  4  Results. 
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Survey  Section  4:  Q.09-19 


Homogeneity  of  variances  (over  study  time  periods)  held  for  all  questions  except  Q12  and  Q13. 
The  following  questions  showed  a  statistically  significant  omnibus  ANOVA: 


Question 

p-value 

N.PRE 

N.Post  1 

N.Post 

Q10 

0.028 

19 

26 

17 

Q12 

<0.0005 

19 

26 

16 

Q13 

0.020 

18 

26 

16 

Q16 

0.025 

18 

25 

17 

The  pairwise  results  follow  and  show  significant  results  between: 

Question 

D-value 

pairwise  studv  time  periods 

Q10 

0.033 

PRE  and  Post  1 

Q12 

<0.0005 

PRE  and  Post  1 

0.003 

PRE  and  Post  2 

Q13 

0.008 

PRE  and  Post  1 

0.048 

PRE  and  Post  2 

Q16 

0.017 

PRE  and  Post  1 

Multiple  Comparisons 


95%  Confidence 

Dependent  Variable 

(1)  Study  Time  Period 

(J)  Study  Time  Period 

Mean 

Difference 

(l-J) 

Std. 

Error 

Sig. 

Inte 

Lower 

Bound 

rval 

Upper 

Bound 

S4_Q10:  The  drug 
information  available 

PRE  (before  BCMA) 

Post  1  (BCMA  stabilization) 

-1  344 

512 

.033 

-2.60 

-.08 

Post  2  (BCMA) 

-1.186 

.566 

121 

-2.58 

21 

through  the  current 
medication  administration 
system  is  easy  to  get 

Bonferroni 

Post  1  (BCMA  stabilization) 

PRE  (before  BCMA) 

1  344' 

512 

033 

08 

2.60 

Post  2  (BCMA) 

.158 

.529 

1.000 

-1.14 

1.46 

when  1  need  that 

Post  2  (BCMA) 

PRE  (before  BCMA) 

1.186 

.566 

121 

-.21 

2.58 

information. 

Post  1  (BCMA  stabilization) 

-.158 

529 

1  000 

-1.46 

1.14 

PRE  (before  BCMA) 

Post  1  (BCMA  stabilization) 

-2.279 

457 

.000 

-3.39 

-1.17 

S4  Q12:  The  current 

Post  2  (BCMA) 

-2.020 

.544 

.003 

-3.38 

-.66 

medication  administration 
system  helps  me  to  be 
efficient  at  medication 

Games- 

Post  1  (BCMA  stabilization) 

PRE  (before  BCMA) 

2.279 

457 

.000 

1.17 

3.39 

Howell 

Post  2  (BCMA) 

.260 

614 

.906 

-1.25 

1.77 

administration. 

Post  2  (BCMA) 

PRE  (before  BCMA) 

2.020 

.544 

.003 

.66 

3.38 

Post  1  (BCMA  stabilization) 

-.260 

614 

.906 

-1.77 

1.25 

S4_Q13:  The  current 
medication  administration 
system  makes  it  easy  to 

PRE  (before  BCMA) 

Post  1  (BCMA  stabilization) 

-1.453 

.456 

.008 

-2.56 

-.34 

Post  2  (BCMA) 

-1  472 

.579 

.048 

-2.93 

-.01 

Games- 

Post  1  (BCMA  stabilization) 

PRE  (before  BCMA) 

1.453' 

.456 

.008 

.34 

2.56 

check  that  1  am  following 
the  “5  rights"  when  1 
administer  medications. 

Howell 

Post  2  (BCMA) 

-.019 

.656 

1  000 

-1.64 

1.60 

Post  2  (BCMA) 

PRE  (before  BCMA) 

1  472' 

579 

.048 

.01 

2.93 

Post  1  (BCMA  stabilization) 

.019 

656 

1.000 

-1.60 

1.64 

PRE  (before  BCMA) 

Post  1  (BCMA  stabilization) 

-1.456 

.504 

.017 

-2.68 

-.23 

34  Q16:  The  current 

Post  2  (BCMA) 

-1.173 

.552 

101 

-2.53 

19 

nedication  administration 
system  is  user-friendly  to 
he  nurses  who  administer 

Games- 

Post  1  (BCMA  stabilization) 

PRE  (before  BCMA) 

1.456' 

.504 

.017 

.23 

2.68 

Howell 

Post  2  (BCMA) 

.282 

.575 

876 

-1.12 

1.69 

dedications. 

Post  2  (BCMA) 

PRE  (before  BCMA) 

1.173 

552 

.101 

-.19 

2.53 

Post  1  (BCMA  stabilization) 

-.282 

.575 

.876 

-1.69 

1.12 

Table  5.  Questions  10, 12, 13  &  16  Statistical  Results. 
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Figure  3.  Graphical  Representation  of  Questions  10, 12, 13  &  16  Results. 

Survey  Section  7:  Q23-29 

Homogeneity  of  variances  (over  study  time  periods)  held  for  all  questions. 

No  questions  showed  a  statistically  significant  omnibus  ANOVA. 

Question  30:  Overall,  how  satisfied  are  you  with  the  current  medication  administration 
system? 

Homogeneity  of  variances  (over  study  time  periods)  held  for  this  question.  This  question 
showed  a  statistically  significant  omnibus  ANOVA,  p-value  =  0.023  and  indicates  a  trend  of 
increasing  dissatisfaction  from  pre  to  post  time  periods.  Figure  4  shows  the  dissatisfaction  at 
MYMC  (rural  facility)  was  slightly  worse  than  the  flagship  hospital,  Memorial,  with  a  greater 
disparity  in  satisfaction  at  MYMC.  Please  note  the  sample  sizes  were  drastically  different. 
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The  pairwise  results  follow: 


Multiple  Comparisons 


Dependent  Variable  (1)  Study  Time  Period  (J)  Study  Time  Period 

Mean 

Difference 

(l-J) 

Std. 

Error 

Sig. 

95%  Confidence  Interval 

Lower 

Bound 

Upper 

Bound 

S_Q30:  Overall,  how 
satisfied  are  you  with 
the  current  medication 
administration  system? 

Bonferroni 

PRE  (before  BCMA) 
N.PRE  =  17 

Post  1  (BCMA  stabilization) 

1.762 

.727 

056 

-.03 

3.56 

Post  2  (BCMA) 

2  082 

.819 

.042 

.06 

4.11 

Post  1  (BCMA  stabilization) 
N,Post  1  =  25 

PRE  (before  BCMA) 

-1.762 

.727 

056 

-3.56 

.03 

Post  2  (BCMA) 

.320 

.755 

1  000 

-1.55 

2.19 

Post  2  (BCMA) 

N?Post  2  =  15 

PRE  (before  BCMA) 

-2  082' 

.819 

042 

-4.11 

-.06 

Post  1  (BCMA  stabilization) 

-.320 

.755 

1  000 

-2.19 

155 

*.  The  mean  difference  is  significant  at  the  0  .05  level 


Table  6.  Question  30  Statistical  Results. 


Q30:  Overall  Satsifaction  with  BCMA 
by  Hospital  over  All  Time  Periods 
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Hospital 

■  Memorial  (main) 
•  MYMC  (off-sile) 


re? ' 


0“ 


PRE  (before 
BCMA) 


Post  1  (BCMA 
stabilization) 


- I - 

Post  2  (BCMA) 


Study  Time  Period 

Error  bars:  95%  Cl 

Figure  4.  Overall  satisfaction  with  BCMA  by  hospital  over  all  time  periods. 


Despite  the  overall  dissatisfaction  of  the  BCMA  system,  a  majority  of  nurses  (71%)  agreed  in 
Figure  5  positively  that  the  system  was  effective  in  reducing  and  preventing  medication  errors. 
These  qualitative  results  will  be  considered  in  regards  to  the  quantitative  analysis  of  error 
counts/rates. 
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The  current  medication  administration  system  is  effective  in  reducing  and  preventing 

medication  errors. 


1  Strongly  Agree 

2  Moderately  Agree  - 

3  Slightly  Agree 

4  Slightly  Disagree 

5  Moderately  Disagree 

6  Strongly  Disagree 

NA  Not  Applicable  - 


Figure  5.  Nurse  opinion  on  BCMA  reducing  and  preventing  medication  errors,  POST  2. 


The  survey  asked  the  respondents  to  input  a  unique  code  ("SUM")  so  that  related-samples 
analysis  could  be  performed  over  the  three  time  periods,  baseline.  Post  1,  and  Post  2. 
Unfortunately,  too  few  respondents  followed  the  instructions  or  were  employed  during  all  time 
points.  Only  six  responses  were  common  between  baseline  and  Post  1,  seven  between  baseline 
and  Post  2,  and  two  over  all  three  time  periods.  This  sample  size  does  not  meet  the  generally 
accepted  threshold  often  for  analysis  to  be  undertaken. 

The  electronic  survey  was  communicated  internally  with  staff  at  MYMC  and  a  link  was  made 
available  on  the  MYMC  Intranet  page  in  early  January  2013  (POST  2).  The  validated  survey 
instrument  by  Hurley  et  al.  (see  Appendix  3)  was  identical  to  the  one  conducted  at  the  flagship 
Memorial  Medical  Center.  A  total  of  17  surveys  were  completed  within  four  weeks,  with  two  of 
those  not  100%  complete.  Data  was  collected  at  three  different  time  points,  PRE  (baseline  - 
2011),  POST  1  (3  months  POST  BCMA  -  2012),  and  POST  2  (8  months  POST  BCMA).  Statistical 
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analysis  was  then  performed  on  the  combined  dataset  over  all  three  time  periods.  Tables  4-6 
and  Figures  2-4  present  the  survey  results  as  indicated.  Sample  sizes  are  generally  as  follows 
but  may  vary  by  question;  N  =  17  (PRE),  N  =  26  (POST  1)  and  N  =  15  (Post  2). 

Please  note  that  the  positively  sloped  trend  lines  in  Figure  2  and  3  indicate  increasing 
dissatisfaction  with  the  BCMA  system  per  the  ordinal  response  scale  used  in  the  survey 
instrument,  refer  to  Appendix  3. 

In  conclusion,  nursing  staff  at  MYMC  is  less  satisfied  with  the  new  BCMA  system  than  the 
previous  manual  medication  administration  system.  The  second  post  survey  results  indicate 
that  satisfaction  continued  to  decline,  which  is  surprising  because  most  users  and  system 
effectiveness  should  be  past  the  learning  curve  phase  at  this  point.  On  the  other  hand,  these 
results  are  not  surprising  because  users  are  likely  a  close-knit  group  located  in  a  small,  rural 
facility  and  may  be  resistant  to  technological  advances.  As  demonstrated  by  the  comments  in 
Table  7,  various  issues  are  present  and  contributing  to  their  dissatisfaction. 

Comments  added  in  response  to  open-ended  survey  questions  during  the  Post  time  periods 
have  been  summarized  in  Table  7  below: 


Issue 

Count 

System  not  user  friendly 

11 

Medication  profile  errors 

6 

Barcodes  not  scanning 

4 

Dangerous  system 

4 

Communication  problems  between  pharmacy  and  nursing 

3 

Medications  not  easy  to  find 

2 

Easier  way  to  check  med  orders 

2 

Medication  info  not  contained  within  application 

1 

Takes  too  long  to  pass  medications 

1 

Timeliness  of  drugs  dispensed 

1 

Table  7.  Summary  of  Comments  Received. 
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Physician  eMAR  Survey  -  CMMC 

Given  the  low  sample  size  (n=9)  from  the  first  announcement,  a  second  announcement  of  the 
survey  was  sent  to  CMMC  physicians,  residents  and  physician  assistants  in  late  November  2012. 
A  modest  response  was  received  during  the  first  quarter,  which  included  30  new  surveys.  The 
current  sample  size  of  39  represents  8%  of  the  medical  and  resident  staff.  Below  are 
descriptive  results  for  a  few  key  questions;  statistical  analysis  is  pending. 


Please  select  your  affiliation. 


Figure  6.  Physician  Affiliation. 


How  often  do  you  access  the  electronic  medication  administration  record  (eMAR)  module 
available  through  Care  Portal? 


Frequently  Sometimes  Seldom  Never 


Figure  7.  Access  to  eMAR  module. 
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The  new  eMAR  module  has  improved  my  decision-making  and  efficiency  becausel  can  now 
electronically  access  if/when  ordered  medications  were  administeredfrom  various  locations. 


Strongly  Agree  Agree  Neutral  Disagree  Strongly  Disagree 


Figure  8.  Improved  decision-making  via  eMAR. 


Adverse  drug  events  have  reduced  because  of  the  bar  coding  technology  andadditional  nursing 
safeguard  warnings/alerts  in  place  to  reduce  medication  errors. 


Strongly  Agree  Agree  Neutral  Disagree  Strongly  Disagree  Don't  Know 


Figure  9.  Physician  perspective  on  reduction  in  adverse  drug  events. 
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Interim  Medication  Error  Analysis  -  CMMC 

The  research  team  completed  intensive  review  of  all  CMMC  medication  events  as  reported  in 
the  Quantros  SRM  event/incident  reporting  system.  Researchers  reviewed  over  1,100  written 
incident  summaries  in  detail  and  logically  determined  the  impact  on  the  MIDHT  study  using  the 
nature  of  the  error  and  when  (phase)  it  occurred  ("Applicability").  The  PRE-BCMA  time  period 
is  November  2010  -  October  2011.  The  POST-BCMA  time  period  is  from  November  2011  - 
January  2013.  The  additional  three  months  was  needed  because  of  the  phased  BCMA 
deployment  schedule  at  CMMC.  All  errors  that  occurred  within  a  unit/department  that  has  not 
deployed  BCMA  technology  were  removed  from  the  dataset. 

The  influence  of  the  MIDHT  intervention  on  medication  errors  from  before  implementation, 
PRE,  versus  after  implementation,  POST,  was  investigated  using  the  Pearson  Chi-squared  test. 
Fisher's  Exact  Test  was  employed  as  required,  e.g.  when  the  count  of  any  cell  in  the  crosstab 
being  analyzed  was  less  than  5.  The  ordinal  variable.  Study  Time  Period,  was  treated  as  the 
independent  variable  (IV).  It  has  only  two  levels,  PRE  and  POST,  both  which  contain  12  months 
of  data  for  each  clinical  unit.  The  statistical  testing  was  applied  in  a  stepwise  manner  relative  to 
the  variables  treated  as  dependent.  The  dependent  variables  are  as  follows: 


Variable  Name 

Variable  Type 

Levels 

Error  Disposition 

nominal 

Near  Miss 

Actual 

MIDHT 

Applicability 

nominal 

Yes 

No 

PHASE 

ordinal 

ORDER  (1st) 

PROFILE  (2nd) 

DISPENSE  (3rd) 

ADMINISTRATION  (4th) 

Other 

Core  Problem 

nominal 

Please  see  Table  10  for  a 

(Med.  Error)  Type 

detailed  listing 

Table  8.  Variable  Name  Description. 
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The  stepwise  analysis  began  with  testing  each  of  the  dependent  variables  separately.  The 
results  were: 


Crosstab 

Chi- 

squared, 

X2 

df 

Sample 

size, 

N 

p-value, 

asymp.  sig. 
(2-sided) 

Error  Disposition  *  Study  Time  Period 

18.43 

1 

1,101 

<  .001  a 

MIDHT  Applicable  *  Study  Time  Period 

4.86 

1 

1,101 

.027  a 

PHASE  *  Study  Time  Period 

11.39 

1 

1,101 

.023 

Core  Problem  (Error)  Type 
*  Study  Time  Period 

180.77 

21 

1,101 

<  .001 

a.  Continuity  correction  applied 

Table  9.  Crosstab  inferential  results. 

Sidak's  correction  for  multiple  testing  was  applied  which  reduced  the  testwise  alpha  criterion  to 
0.017.  This  correction  is  slightly  conservative  in  that  no  correlation  is  assumed  to  exist  among 
the  variables.  A  correlation  analysis  of  (between)  the  dependent  variables  revealed  an  average 
correlation  of  0.1,  which  rises  slightly  to  0.3  if  only  the  statistically  significant  correlations  are 
averaged.  Including  the  0.3  correlation  into  the  calculation  of  Sidak's  correction  yields  a 
testwise  alpha  criterion  to  0.019  and  corresponding  critical  z-value  for  2  sided  testing:  >= 

2.3407  (critical  z-value  for  1  sided  testing:  >=  2.0695).  Therefore,  only  Error  Disposition  *  Study 
Time  Period  and  Core  Problem  (Error)  Type  *  Study  Time  Period  crosstabs  have  a  statistically 
significant  result. 


Core  Problem  (Med  Error)  Type 

Bar  Coding/Scanning  Error 

Charting  Error 

Drug  Delay 

Drug  Location  Issue 

Drug  Not  Ordered 

Drug  Omitted 

Drug  Protocol/Policy  Not  Followed 

Incorrectly  Stamped  Order  Sheet 

IV  Related  Issue 

Medication  Reconciliation  Issue 

Order  Issue 

Patient  Not  Compliant 

Pharmacy  Profile  Error 

Pyxis  Issue 

Wrong  Dose 

Wrong  Drug 

Wrong  Frequency 

Wrong  Label 

Wrong  Patient 

Wrong  Quantity 

Wrong  Route 

Table  10.  Medication  Error  Type  Listing. 
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These  interim  results  indicate  that  Error  Disposition  and  Study  Time  Period  are  not  statistically 
independent  over  the  entire  dataset  of  medication  errors;  therefore  differences  in  error 
disposition  are  related  to  the  difference  in  Study  Time  Period.  Standardized  residuals  of  the 
crosstab  revealed  that  cells  of  the  levels  of  Near  Miss  for  both  Pre  and  Post  contributed  to  the 
significant  chi-square  result.  Collectively,  this  implies  that  the  total  of  Near  Miss  errors 
(whether  MIDHT  Applicable  or  not)  increased  from  PRE  to  POST.  The  standardized  residuals  for 
Actual  errors  indicate  no  contribution  to  the  chi-square  significance,  which  implies  no 
statistically  significant  difference  between  time  periods.  Given  this  the  statistically  significant 
result  for  the  crosstab  of  Error  Disposition  *  Study  Time  Period,  a  more  detailed  analysis  was 
performed  on  a  stratified  crosstab  of  Error  Disposition  by  MIDHT  Applicability  versus  Study 
Time  Period.  No  statistical  significance  was  found  for  MIDHT  Applicability  on  either  level  of 
Error  Disposition. 

The  interim  results  indicate  that  the  distribution  of  Core  Problem  (Error)  Type  differs  over  Study 
Time  Period.  The  standardized  residuals  reveal  the  following  variables  contribute  to  that 
difference: 


Implication  regarding  change 
from  PRE  to  POST 

Drug  Delay 

decrease 

Order  Issue 

increase 

Incorrectly  Stamped  Order  Sheet 

increase 

Note  that  the  implications  drawn  from  the  Core  Problem  (Error)  Type  *  Study  Time 
Period  chi-squared  analysis  are  with  caution  because: 


•  10  cells  (22.7%)  of  the  22  by  2  crosstab  have  an  expected  count  less  than  5 

•  14  cells  have  an  observed  count  <  5 

•  14  cells  have  an  observed  count  <  10 

Table  11.  Standardized  residual  results. 
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Pyxis  Issue' 
Drug  Location  Issue' 
Wrong  Quanti 
Bar  Coding/Scanning  Erro 
Wrong  Labe 
Drug  NotOrdere 
Patient  Not  Complian 
Wrong  Rout 
Drug  Protocol/Policy  NotFollowe 
Medication  Reconcilliation  Issue' 
Charting  Erro 
Pharmacy  Profile  Erro 

IV  Related  Issue' 
Wrong  Frequenc 
Wrong  Patien 
Order  Issue 
Wrong  Dru 
Wrong  Dose 
Incorrectly  Stamped  Order  Shee 
Drug  Dela: 
Drug  Omitte 


Count 


Study 

Time 

Period 

■  pre 
□  post 


Figure  10.  Medication  Error  Type  by  Period. 


Common  errors,  such  as  wrong  patient,  wrong  drug,  and  wrong  dose  all  had  a  somewhat 
neutral  change  from  PRE  to  POST.  Either  the  frequency  of  each  medication  error  type  is  so  low 
that  no  significant  change  is  detected  or  the  error  can  be  attributed  to  human  error  and 
therefore  not  affected  between  time  periods.  The  culture  of  reporting  errors  had  changed  in 
the  POST  period,  which  complicates  this  analysis. 


The  medication  administration  process  is  not  simply  the  act  of  administering  medication,  but 
rather  complex  processes  involving  multiple  steps  from  numerous  disciplines,  departments,  and 
users.  Errors  can  appear  at  one,  some,  or  even  all  stages  between  the  medication  order 
process  (prescription)  through  the  actual  administration  of  the  medication  and/or  possible 
adverse  events.  Though  a  specific  error  may  occur  prior  to  a  patient  receiving  the  medication, 
most  errors  are  noted  only  once  the  medication  is  incorrectly  administered  at  the  bedside. 
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Despite  the  fact  that  BCMA  strives  to  assure  the  "five  rights"  of  medication  administration  - 
right  patient,  right  route,  right  drug,  right  dose,  and  right  time,  most  studies  however  have 
investigated  the  effect  of  BCMA  on  the  rate  of  severity  of  the  errors  or  the  effect  of  the  BCMA 
on  the  duration  of  administering  medication. 

Medication  errors  can  vary  depending  on  the  differing  levels  of  care  per  hospital  floor.  A  recent 
literature  review  study  1(Hassink,  2012)  reports  average  baseline  error  frequency  rates  from 
5.8-25.3%  (1.6-27.8%  if  error  times  were  excluded).  Wrong  time  errors  are  generally 
considered  to  be  less  severe.  Typical  reports  after  BCMA  implementation  include  a  30-50% 
reduction  in  medication  errors,  but  only  when  time  errors  are  excluded.  For  this  interim  data 
analysis  medication  errors  are  only  listed  as  counts,  rather  than  error  rates.  The  final  study 
report  will  include  more  detailed  and  more  comparable  analyses  on  rates. 

The  severity  of  drug  errors  will  also  be  addressed  in  the  final  study  report.  Some  studies  have 
reported  a  nearly  50%  reduction  in  risks  post  BCMA  implementation  2  3  4(Poon,  2010;  Franklin, 
2007;  Morris,  2009).  Error  disposition  (i.e.  near  miss  or  actual  miss)  per  time  periods,  nursing 
unit,  and  other  appropriate  delineations  may  lead  to  subsequent  conclusions  about  BCMA 
implementation.  Initial  discussions  were  rooted  in  the  possibility  that  reporting  frequency  of 
actual  vs  near  miss  may  differ  (e.g.  near  miss  may  be  reported  less  frequently  to  save  time;  or 
near  miss  errors  may  be  reported  more  POST  BCMA  implementation  because  nurses  were 
disgruntled/dissatisfied  with  the  technology  and/or  disruptions  to  their  traditional  workflow, 
PRE  implementation).  The  final  study  report  will  include  more  detailed  and  comparable 
analyses  on  error  severity. 

Most  medication  error  research  however  has  been  completed  in  small  units  and  thus  for 
comparative  purposes  that  final  data  analyses  may  examine  the  impact  of  BCMA  on  specifically 
identified  nursing  units  at  CMMC.  Differences  between  nursing  units  must  be  considered 
because  of  nurse-to-patient  ratio,  as  well  as  hospital  census  differentials. 

Work-arounds  performed  by  nursing  staff  must  be  acknowledged  as  a  contributing  factor  on 
the  medication  error  rates.  The  following  table  demonstrates  manual  override  statistics  from 
CMMC  for  July  through  September  2013.  Though  these  reporting  statistics  are  not  available 
during  the  study  time  period,  researchers  will  utilize  this  data  to  make  inferences  to  the  impact 
on  medication  errors  by  clinical  unit. 


McK  Adoption  HARx  Analytics  Medication  Administration 
July  -  September,  2013  (3-month  Average)  N=  8,525  medications  given 


Schedule 

Override 

Count 

%  w 

Schedule 

Override 

Drug  Warning 
Override  Count 

%  w  Drug 
Warning 
Override 

Quantity 

Override 

Count 

%  w 

Quantity 

Override 

83 

2.96 

79 

2.86 

39 

0.62 

Table  12.  CMMC  manual  overrides  by  type  (3-month  average). 
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Considerations  for  future  research  are  to  see  if  medication  errors  could  be  segregated  based  on 
origination  (i.e  if  drug  dispensed  from  the  robot  vs  the  MedCarousel  vs.  manual  picks  results  in 
any  differences  in  the  number,  type  etc.  of  medication  errors).  No  past  studies  investigated 
user  compliance  in  the  BCMA  system  (ie.  work-arounds  with  the  BCMA  system).  Such  may 
need  to  be  re-evaluated  over  time  since  as  system  familiarity  evolves,  so  could  the  work¬ 
arounds.  Longer-term  follow-up  studies  (>  2  years)  are  needed  both  for  the  aforementioned 
reason  and  also  for  examination  of  the  degree  and  time  post  implementation. 

Nursing  Time  and  Motion  Final  Results  -  CMMC  &  MYMC 

The  primary  goal  of  the  BCMA/eMAR  implementation  was  to  increase  patient  safety  -  hopefully 
not  at  the  expense  of  procedural  efficiency.  A  time  and  motion  tool  was  employed  to  capture 
data  capable  of  providing  insight  into  changes  of  that  efficiency  for  nursing  (and  pharmacy). 

Study  team  personnel  physically  shadowed  the  clinician  (RN  or  LPN)  during  multiple  contiguous 
four-hour  blocks  of  time  during  their  work  shift  before  (PRE)  and  after  (POST)  BCMA/eMAR 
implementation.  Prior  to  shadowing,  the  study  team  met  with  the  clinicians  to  confirm  the 
completeness  and  accuracy  of  the  activities  listed  on  the  time  and  motion  tool,  the  activity 
(AID)  index,  as  sufficient  representations  of  the  corresponding  actual  workday  activities.  Effort 
was  made  to  obtain  a  representative  data  sample  by  varying  the  clinician,  the  hospital  floor 
(e.g.  patient  type  and  load),  and  time  of  day  of  the  observation.  As  used,  the  time  and  motion 
tool  consisted  of  35  unique  activities  (AIDs).  The  implementation  of  BCMA/eMAR  was  expected 
to  alter  nursing  procedures  particularly  at  the  task  (activity)  level.  Some  tasks  would  be 
unaffected,  some  would  be  expanded  in  scope  while  others  would  be  reduced,  and  a  few  would 
be  eliminated.  These  procedural  adjustments  introduced  variation  from  PRE  to  POST  into  some 
AIDs.  The  confounding  influence  of  this  variation  was  managed  first  through  the  defining  of  the 
AIDs  so  as  to  minimize  the  replacements  from  PRE  to  POST  shadowing,  and  second  by  the 
categorization  of  the  AIDs  into  work  processes.  The  presumption  being  that  the  magnitude  of 
any  change  at  the  AID  level  would  not  be  so  large  as  to  change  the  process  category  of  that 
activity.  For  analysis,  each  AID  was  placed  into  one  often  processes  (PIDs);  these  processes 
represent  collections  of  activities  of  related  function,  see  Table  13.  Nurse  Shadowing:  Process 
(PID)  Details1. 

Every  time  study  personnel  observed  and  recorded  an  activity,  it  represented  an  instance  of 
that  AID.  When  the  observed  task  was  not  represented  by  a  predefined  AID,  that  instance  was 
recorded  using  the  'other'  AID.  The  time  and  motion  tool  recorded  the  elapsed  time  per  each 
instance  of  an  AID.  The  sum  of  the  duration  (time)  of  all  the  instances  of 
AIDs  yielded  a  total  observation  time  per  time  period.  Similarly,  the  sum  of  the  time  per 
instance  of  the  AIDs  assigned  to  a  PID  yielded  the  total  time  per  PID.  The  total  time  of 
shadowing  PRE  and  POST  for  each  location  was  nearly  equal  (Table  13).  A  normalization 
procedure  was  therefore  applied  to  allow  for  legitimate  direct  comparisons  of  the  data 
between  time  periods  (by  location  -  CMMC  and  MYMC).  The  total  time  for  each  location  by 
time  period  (PRE  and  POST)  was  calculated  and  used  to  normalize  (rescale)  the  data.  The 


1  After  further  discussion,  AID  19  was  moved  from  PID  3  to  2;  AID  20  from  PID  3  to  1 1 
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elapsed  time  for  each  instance  of  AID  was  divided  by  the  total  time  for  each  location.  The 
normalized  data  was  then  multiplied  by  100  to  yield  the  outcome  variable  of  analysis, 
dTime_normd_pct. 


Total  Time 

per  Location  and  Time  Period 


Time 

Period 

MMC 

MYMC 

hh  mm  ss 

seconds 

Observations 

hh  mm  ss 

seconds 

Observations 

PRE 

105:22:10 

379,330 

26 

3:55:00 

14,100 

i 

POST 

104:53:31 

377,611 

26 

4:15:58 

15,358 

i 

difference 

between 
time  periods 

0:28:39 

0:20:58 

Table  13.  Nurse  Shadowing  Total  Time 

Note:  seconds  bas  e  been  provided  to  ease  the  conversion  of  percentages  and 
normalized  data  found  in  the  text  into  units  of  time 


This  derived  variable  represents  the  percent  of  total  time  (per  location  per  time  period)  for 
each  instance  of  AID. 

The  between  time  period  analysis,  PRE  vs.  POST,  for  each  location  was  applied  on  the  following 
levels  of  stratification: 

1.  process (PID)  and 

a.  BCMA/eMAR  (and  related)  activities 

2.  individual  activity  (AID) 

3.  intra-location  geography  by  PID 

A  PID  is  a  stratification  of  the  data  by  aggregating  every  instance  of  specific  AIDs  of  related- 
function.  Most  PIDs  contain  multiple  AIDs. 

The  BCMA/eMAR  level  of  analysis  focuses  on  a  subset  of  AIDs  of  PID  1;  specifically  AIDs  4,  5, 14, 
and  24.  This  level  of  analysis  was  necessary  since  AIDs  4,  5,  and  14  were  directly  impacted 
(replaced)  by  AID  24  (BCMA/eMAR,  POST  only).  In  fact,  the  data  showed  that  AIDs  4,  5,  and  14 
were  not  used  during  the  POST  shadowing  and  therefore  effectively  eliminated.  The  tasks 
represented  were  now  being  accomplished  by  the  BCMA/eMAR  systems.  Therefore,  this 
analysis  compared  the  aggregation  of  4,  5,  and  14  in  the  PRE  with  all  instances  of  AID  24  (POST). 
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Activity 

ID 

(AID) 

Process 

ID 

(PID) 

Process 

Description 

Individual  Activity  Description 

Process 

ID 

1 

Passing  medications 

1 

2 

Hang  IV fluids 

1 

3 

Retrieve  meds  from  Pyxis,  bins,  or  tube 

1 

4 

Compare  physical  med  to  paper  MAR 

1 

5 

1 

MED 

Confirm  "Five  Rights" 

1 

7 

administration 

Passing  IM/Subq  meds 

1 

8 

Passing transdermal  meds 

1 

9 

Stock  or  Retrieve  from  MED  cart 

1 

14 

Transcibe  med  orders  -  paper  MAR 

1 

24 

eMAR 

1 

11 

SCANning 

2 

19 

computer 

charting 

review  lab/rad  results  -  computer 

2 

21 

2 

Admissions 

2 

22 

Discharges 

2 

23 

Standard 

2 

51 

patient-related 

activities 

Pt  assess ment-e.g.  vitals 

3 

52 

3 

Physician  Rounding-listening 

3 

53 

Pt  assistance-e g.  meals 

3 

64 

talking  in-person 
to::> 

Colleague/Staff  for  Pt. 

4 

66 

4 

verbal  report-out  with  nurses 

4 

71 

Patient/Family 

4 

76 

5 

MED-related  interruptions  ONLY 

5 

73 

6 

waiting 

waiting  for  MED  cart 

6 

74 

waiting  for  Pxyis  access 

6 

72 

7 

walking  inside 

7 

79 

Patient  (patient  family) 

8 

80 

Doctor 

8 

81 

8 

Phone 

Nurse 

8 

82 

Pharmacy 

8 

85 

misc. 

8 

20 

fax,  copy 

11 

55 

0)1 

Paper 

check  or  write  PAPER  charts 

11 

56 

review  lab/rad  results  -  PAPER 

11 

70 

Nurse's  note  for  charting  later 

11 

62 

10 

personal  time-Any 

10 

Table  14.  Nurse  Shadowing:  Process  (PID)  Details  . 

The  AID  analysis  level  is  the  aggregation  of  every  instance  of  each  AID  used  to  create  the  PIDs. 
Intra-location  geography  is  a  stratification  by  unit  within  CMMC,  namely: 

•  A9  =  Ashman/Rose  Pavilion,  floor  9,  Ashman  section 

•  RIO  =  Ashman/Rose  Pavilion,  floor  10,  Rose  section 

•  GS5  =  Good  Samaritan  Building,  floor  5 

•  MS7  =  Medical  Surgical  7 

These  floors  were  chosen  as  a  representative  sample  of  CMMC  based  upon  physical  layout  and 
patient  composition.  Given  the  size  of  MYMC  (20-bed  rural  hospital),  the  intra-location 
geography  concept  is  not  applicable. 

ANOVA  analyses,  between  time  period  (PRE  vs.  POST),  was  accomplished  for  each  location 
(CMMC  and  MYMC)  separately.  Applying  an  ANOVA  analysis  to  dTime_normd_pct  provides  the 
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most  effective  manner  in  which  to  simultaneously  account  for  both  intrinsic  characteristics  of 
this  dataset,  sum  of  time  and  frequency,  that  influence  the  quantification  of  efficiency 
(productivity)  and  gross  system  changes.  Analysis  of  the  mean  of  dTime_normd_pct  accounts 
for  both  components  and  provides  directly  comparable  results  across  time  periods.  A  family- 
wise  alpha  =  0.05  was  set  as  the  criteria  for  statistical  significance. 

The  statistically  significant  finding  on  the  BCMA/eMAR  level  implies  that  the  implementation  is 
being  utilized  as  intended.  Also,  since  no  PID  individually  and  only  one  AID  individually,  which 
likely  would  not  be  influenced  by  BCMA  implementation,  showed  statistical  significance,  the 
results  imply  that  the  implementation  of  bar  coded  medication  administration  did  not  positively 
or  negatively  impact  the  efficiency  of  the  nursing  workflow  at  CMMC  as  analyzed.  Please  refer 
to  Figure  11  and  Tables  19  and  20  (Appendix  4)  for  details. 


Nurse  Shadowing:  Summary  of  Percentages 
Location  =  CMMC 


■■PRE  ■■POST  -®- cumulative %,  PRE  —A — cumulative %,  POST 


100 % 
90% 
80% 
70% 
60% 
50% 
40% 
30% 
20% 
10% 
0% 


(PID):  Description 


Figure  11.  Nurse  Shadowing:  Summary  of  Percentages,  CMMC 

Note:  Cumulative  percent  does  not  reach  100%  because  the  "Other"  PID  is  not  graphed  since  these  activities  are  not 
germane  to  the  analyses. 
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PRE  vs  POST 

PID 

n/PRE 

n/POST 

p-value 

Change  of  the  meanP0ST  relative 

to  that  of  the  PRE 

PIDs,  individually 

No  statistical  significance  was  found. 

BCMA/eMAR 

1 

(subset  of) 

432 

681 

<0.0005 

Increase 

Intra¬ 

location 

geography 

by 

PID 

A9 

1 

2 

475 

124 

512 

182 

0.013 

0.005 

Increase 

Decrease 

GS5 

No  statistical  significance  was  found. 

MS7 

4 

7 

269 

229 

264 

284 

0.045 

0.027 

Decrease 

Decrease 

RIO 

1 

270 

342 

0.009 

Increase 

AIDs,  individually 

AID  3 

1 

238 

179 

<  0.0005 

Decrease 

Table  15.  Summary  of  Statistical  Findings  -  CMMC 


Also,  for  CMMC,  analyses  were  conducted  on  average  daily  census  (ADC),  case  mix  index  (CMI) 
and  between  intra-location  geography.  PRE  was  compared  with  POST.  Given  the  sample  size 
for  the  PRE  time  period,  Mann-Whitney  test  was  used.  No  statistical  significance  was  found 
overall  for  the  analysis  by  time  period.  The  data  set  was  stratified  by  intra-location  geography 
and  repeated.  No  statistical  significance  was  found.  The  between  intra-location  geography 
ANOVA  produced  a  significant  contrast  only  between  A9  and  GS5  (p-value  =  0.029)  indicating 
that  these  two  floors  differ  from  each  other  on  dTime_normd_pct.  No  evidence  was  found  to 
imply  that  this  result  impacted  the  conclusions  drawn  overall. 

MYMC: 

Statistical  significance  was  found  for  two  groupings,  'MED  administration'  and  'Talking  in- 
person',  but  no  individual  activities.  Statistical  significance  was  found  for  BCMA/eMAR  level  as 
well,  implying  that  the  implementation  is  being  successfully  utilized  as  intended.  The  findings 
must  be  viewed  with  caution  since  the  data  contains  observations  from  only  one  clinician  for 
one  four-hour  time  period,  which  is  insufficient  to  be  a  representative  sample;  an  unavoidable 
consequence  of  the  typically  low  patient  census  at  this  location.  If  the  significance  would  hold 
for  a  more  representative  sample,  the  implication  could  be  that  BCMA  has  some  influence  at 
MYMC.  Please  refer  to  Figure  12  and  Tables  21  and  22  (Appendix  4)  for  details. 
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Nurse  Shadowing:  Summary  of  Percentages 
Location  =  MYMC 


(PID):  Description 


Figure  12.  Nurse  Shadowing:  Summary  of  Percentages,  MYMC 

Note:  Cumulative  percent  does  not  reach  100%  because  the  "Other"  PID  is  not  graphed  since  these  activities  are  not 
germane  to  the  analyses. 


PRE  vs  POST 

PID 

n»PRE 

O'POST 

p-value 

Change  of  the  meanly  relative 

to  that  of  the  PRE 

PIDs, 

1 

35 

22 

0.001 

Increase 

individually 

4 

16 

28 

0.03 

Decrease 

BCMA/eMAR 

1 

(subset  of) 

24 

12 

<  0.0005 

Increase 

Intra-location  geography 

Not  Applicable 

AIDs,  individually 

No  statistical  significance  was  found. 

Table  16.  Summary  of  Statistical  Findings  -  MYMC 
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Pharmacy  Time  and  Motion  Study  -  CMMC 

The  research  team  completed  time  and  motion  observations  of  pharmacists  and  technicians 
starting  the  week  of  April  22nd.  This  data  collection  occurred  after  the  implementation  of  the 
second  MedCarousel.  A  total  of  12  observations  were  performed  to  match  baseline  data 
collection,  including  the  same  day(s)/time(s)  of  the  week  and  similar  time  of  year  (seasonality). 
Each  subject  was  shadowed  for  a  4-hour  period  as  detailed  in  the  table  below. 


Date 

Time 

Position 

April  22,  2013 

7:00-11:00  AM 

Pharmacist 

April  23,  2013 

9:00  PM  -1:00  AM 

Pharmacist 

April  24,  2013 

8:30  AM -12:30  PM 

Technician 

April  29,  2013 

12:00-4:00  PM 

Pharmacist 

April  30,  2013 

9:00  PM  -1:00  AM 

Technician 

May  2,  2013 

2:30-6:30  PM 

Pharmacist 

May  8,  2013 

10:00  AM -2:00  PM 

Technician 

May  13,  2013 

11:00  AM -3:00  PM 

Technician 

May  15,  2013 

3:00-7:00  PM 

Technician 

May  17,  2013 

8:30  AM-12:30  PM 

Technician 

May  20,  2013 

7:00-11:00  AM 

Technician 

May  24,  2013 

7:00-11:00  AM 

Technician 

Table  17.  Pharmacy  Time  &  Motion  Schedule. 

The  following  chart  displays  descriptive  results  per  process  grouping. 


Percent  of  Total  Time  by  Process 


50% 

4S% 


Figure  13.  Percent  of  Total  Time  by  Process  (PRE  VS.  POST). 
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Figure  14.  Total  Time  per  Process  (PRE  VS.  POST). 

The  infusion  of  bar  code  technology  into  the  pharmacy  at  CMMC  had  significant  impact  on 
staff,  processes,  and  the  overall  atmosphere  of  the  department.  Upon  preliminary  examination 
of  the  data  collected  in  2011  (PRE)  compared  with  data  collected  in  2013  (POST),  multiple 
workflow  process  alterations  have  been  noted.  Information  regarding  patient  census,  staffing 
levels,  process  changes,  etc  will  be  considered  for  the  final  study  report. 

An  expected  benefit  of  automation  is  the  consolidation  of  inventory  in  known  locations  to  staff. 
Pharmacy  technicians  spent  nearly  the  same  amount  of  time  filling  medication  orders  between 
the  PRE  and  POST  time  periods.  This  can  be  considered  a  positive  result  of  the  new  technology, 
as  the  automation  of  processes  did  not  increase  the  time  for  drug  dispensation.  However, 
efficiency  gains  were  not  quantitatively  realized  for  this  process.  (Discussion  as  to  the  possible 
reasons  for  this  result  will  be  examined  during  the  more  in-depth  data  analyses.)  The  results  do 
show  a  decrease  in  the  amount  of  time  spent  crediting  inventory;  likely  due  to  the  automation 
process  now  employed  to  complete  this  task  and/or  the  distribution  of  the  prescribed  quantity 
and  type  of  drug(s)  dispensed  to  the  floor. 

A  reduction  in  phone  calls  and  in  handling  paper  (Fax,  Print,  Copy,  Notating)  has  been  shown 
during  the  POST  period.  This  implies  fewer  calls  from  the  floor  (e.g.  nurse)  were  received 
questioning  the  status  of  ordered  medications  as  well  as  less  hand-off  communications  (i.e. 
paper  handling).  A  reduction  in  physical  walking  has  also  been  realized,  implying  technicians 
are  spending  less  time  traversing  to  fill  medication  orders  and  is  also  demonstrative  of  the 
overarching  role  of  the  robotic  technology  -  to  fill  patient  prescription  orders  via  automation. 
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The  increase  in  personal  time  (e.g.  meal  break)  during  the  POST  assessment  can  be  seen  as  a 
positive  outcome  from  the  perspective  of  employee  satisfaction  and  perhaps  increased  task 
completion  efficiency,  as  pharmacists  and  technicians  are  seemingly  able  to  more  consistently 
take  scheduled  breaks  during  shifts. 

Initial  examination  of  the  data  also  revealed  the  following: 

•  A  small  decrease  in  time  spent  for  order  entry.  This  is  expected  because  this  process 
was  not  primarily  impacted  by  technology  upgrades  or  automation.  Computerized 
physician  order  entry  (CPOE),  currently  being  piloted  by  a  small  group  of  CMMC 
physicians  (not  included  in  MIDHT  scope)  was  anticipated  to  increase  efficiencies,  but 
was  rarely  observed  during  workflow  shadowing. 

•  A  small  increase  in  time  spent  checking  medications.  This  result  is  surprising  because 
procedurally  only  10%  of  medications  dispensed  from  the  robot  are  checked  by  a 
pharmacist  (POST)  whereas  100%  of  all  medications  manually  filled  by  technicians  were 
reviewed  in  the  PRE  period. 

School  of  Nursing  Bar  Code  Medication  Administration  (BCMA)  Survey 

An  abbreviated  BCMA  baseline  survey  was  distributed  to  all  nursing  students  on  December  3, 
2012.  A  strong  response  was  received  (n=102)  before  the  survey  was  closed  on  December  14, 
2012.  Survey  respondents  that  followed  instructions  received  a  token  incentive  for  completing 
a  valid  survey.  Five  respondents  stated  in  written  comments  that  they  would  like  improved 
access  to  medication  carts  for  training  purposes. 

A  total  of  87  Conemaugh  nursing  students  completed  the  POST  BCMA  survey  during  April  and 
May  of  2013,  which  included  a  good  balance  between  1st  and  2nd  year  students.  The  following 
select  questions  suggest  a  positive  change  in  access  to  the  system  after  additional  medication 
carts  were  made  available  and  a  favorable  opinion  of  the  BCMA  system  on  improving  patient 
safety. 
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Q5  Current  status: 


Answered:  87  Skipped:  0 


Figure  15.  Current  status  of  nursing  student. 


Do  you  have  access  to  medication  carts 
(blue  Rubbermaid  with  drawers)  when  you 
need  them  for  training  on  the  floor? 

Answerer);  81  SKipi"ni,  v 


100% 

80% 

60% 


Frequently  Very  Often  Sometimes  Not  Very  Never 

Often 


Figure  16.  Access  to  medication  carts  -  POST. 

Comparison  of  mean  statistics  between  PRE  (Sometimes  -  2.75)  and  POST  (Very  Often  -  3.21) 
results  suggest  a  positive  change  occurred  through  the  purchase  of  additional  medication  carts 
for  the  School  of  Nursing.  Ultimately,  a  favorable  training  experience  was  created  through 
improved  access  to  the  BCMA  system. 
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;  I  The  current  medication  administration 
system  is  effective  in  reducing  and 
preventing  medication  errors. 

Airswere>J:  (H  Skipped:  *j 
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Figure  17.  Impact  on  preventing  medication  errors. 

Nursing  students  did  not  suggest  any  real  change  in  patient  safety  deduced  from  the 
same  median  and  mode  for  both  time  periods  (Strongly  Agree).  Overall,  they  believe 
the  BCMA  system  is  effective  is  preventing  medication  errors  but  additional  system 
usage  did  not  change  their  opinion. 


Overall,  how  satisfied  are  you  with  the 
current  medication  administration  system? 
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Figure  18.  Overall  satisfaction  of  BCMA  system. 

Overall  satisfaction  did  not  change  over  time  with  a  median  of  9.0  for  both  PRE  and  POST  data 
sets.  Nursing  students  have  a  favorable  opinion  of  the  BCMA  system  and  likely  cannot  compare 
historically  to  the  manual  medication  administration  system  previously  used  at  Conemaugh. 
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Arm  2:  Health  Information  Exchange  (HIE)  via  the  Nationwide  Health  Information  Network 
(NwHIN) 

Subtask  2.1  Deploy  a  limited  production,  NHIN  standards-based  HIE  focusing  on  the  bi¬ 
directional  exchange  of  electronic  medical  records  between  CHS  and  the  Military  Health 
System.  CHS  information  to  include  data  domains  residing  in  acute  care  and  ambulatory 
settings. 

Conemaugh  Activities 

The  initial  Virtual  Lifetime  Electronic  Record  (VLER)  mass  mailing  was  sent  to  355  veterans  on 
October  1st  and  2nd.  For  the  first  quarter,  42%  of  patients  decided  to  participate  in  the  program. 
The  mailing  included  a  joint  letter  (see  Appendix  5),  two  consent  forms  and  marketing 
materials.  Conemaugh  handled  all  administration  aspects  of  the  mailing. 

Once  a  Veteran  returns  completed  consent  forms,  MIDHT  staff  "opt-in"  that  specific  patient 
through  the  CONNECT  Consumer  Preferences  Profile  (CPP).  VA  forms  are  provided  to  them  on 
a  weekly  basis  as  needed.  Patient  correlations  are  checked  frequently  to  determine  status  of 
linkages. 

Once  a  correlation  is  made  between  Conemaugh  and  the  VA,  MIDHT  staff  adds  chart  alert 
within  the  Allscripts  electronic  health  record  to  notify  providers  of  the  event.  This  process  was 
implemented  in  mid-November. 
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Figure  19.  Adscripts  Chart  Alert. 

Conemaugh  staff  completed  refresher  training  with  Portage  Health  Center  on  October  31,  2012 
and  Ebandjeff  Health  Center  on  November  14,  2012.  New  system  users  are  added  to  the 
OpenDS  directory  service  as  needed  and  MIDHT  staff  also  adds  them  to  the  "NwHIN  Users 
Group"  on  a  regular  basis  so  they  can  access  the  CONNECT  application  via  Care  Portal. 
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An  initial  meeting  was  held  with  Dr.  Lieb  and  St.  Benedict  staff  on  December  11,  2012,  in  which 
the  VA  Exchange  project  was  detailed.  The  meeting  went  well  and  they  were  excited  to 
participate.  Invitations  to  known  shared  patients  (n=62)  were  mailed  on  December  26,  2012. 

Conemaugh  staff  assisted  Northrop  Grumman  Corporation  (NGC)  during  the  month  of 
December  with  the  MIDHT  server  move  to  Chester,  VA,  including  MPI,  CAL,  VPN  and  Care  Portal 
configuration  changes.  Conemaugh  worked  with  HealtheWay  to  update  the  UDDI  registry  with 
new  GO-LIVE  endpoints.  The  Conemaugh  gateway  had  a  downtime  of  1.5  business  days. 

Conemaugh  participated  in  VA  testing  activities  on  December  5,  2012.  VA  completed  an 
adapter  upgrade  and  we  confirmed  that  the  service  was  operational  and  that  numerous  C32 
changes  were  reflected  through  xml  examination. 

Project  team  identified  an  issue  where  medication  data  (from  Allscripts)  for  some  patients  was 
not  being  populated  in  the  CAL  response  and  therefore  the  C32  clinical  document.  Issue  was 
resolved  on  March  27,  2013  and  verified  accurate  through  testing. 

Conemaugh  has  been  working  with  Sunquest  Information  Systems,  Inc.  to  discuss  an  on- 
demand  HL7  v2  interface  for  hospital-based  lab  results.  A  quote  was  prepared  for  client  review. 
The  addition  of  lab  results  to  the  Conemaugh  C32  has  been  requested  by  Altoona  VAMC 
stakeholders  and  will  support  coordination  of  care  and  quality  initiatives.  Conemaugh  had  been 
working  with  Sunquest  Information  Systems,  Inc.  to  stand  up  an  on-demand  HL7  v2  interface 
for  hospital-based  lab  results.  A  purchase  order  was  executed  on  April  24th  and  sent  to  the 
vendor.  Conemaugh  has  been  frustrated  by  the  lack  of  movement  by  Sunquest  on  this  project. 
A  project  schedule  was  finally  provided  to  CMMC  on  August  7,  2013,  based  upon  availability  of 
Sunquest  resources.  The  project  was  expected  to  begin  in  early  September  but  has  been 
further  delayed  by  the  vendor. 

Response  by  Veterans  to  the  VLER  mailing  invitations  continues  to  be  strong.  Table  18 
summarizes  various  efforts  during  the  quarter  to  increase  the  number  of  authorizations  (i.e. 
patients  opted  in),  including  patients  of  CPG  St.  Benedict  and  a  "Final  Notice"  mailing  to 
patients  (occurred  in  mid-February)  of  the  initial  pilot  group.  John  Hargreaves  met  with  four 
Conemaugh  primary  care  practices  during  the  month  of  March  to  discuss  the  purpose  of  VLER 
and  HIE.  System  training  was  also  completed  with  staff,  showing  them  how  to  request 
documents  from  the  VA.  New  CONNECT  accounts  were  created  in  conjunction  with  user  setup. 
We  also  have  approached  "shared  patients"  that  do  not  have  a  primary  care  provider  (PCP), 
patients  with  VA  PCP's  and  patients  with  an  independent  PCP. 
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Item 

#  of  veterans 

#  of  veterans  that 

#  of  veterans 

contacted 

responded 

correlated 

CPG  St.  Benedict 

63 

40 

(63%) 

39 

CPG  East  Hills 

74 

43 

(58%) 

43 

CPG  Portage 

140 

46 

(33%) 

46 

CPG  Ebandjeff/Park  Hill 

119 

53 

(45%) 

53 

CPG  Berkley  Hills 

55 

35 

(64%) 

35 

No  PCP 

463 

101 

(22%) 

99 

CPG  Highlands 

49 

19 

(39%) 

18 

CPG  Dr.  Pote 

23 

12 

(52%) 

12 

CPG  Dr.  Gvozden 

50 

15 

(30%) 

14 

CPG  Drs.  Massoud  and  Sirsikar 

72 

22 

(31%) 

22 

CPG  Dr.  Johns 

67 

19 

(28%) 

18 

VA  Providers 

58 

14 

(24%) 

14 

Dr.  Munzer 

32 

13 

(41%) 

12 

Dr.  Trostle 

18 

4 

(22%) 

4 

Other  Independent  Practices! 

400 

31 

(8%) 

4 

Face  to  face 

43 

43 

(100%) 

43 

TOTAL  DURING  YEAR 

1,726 

510 

(28%) 

476 

GRAND  TOTAL 

1,743 

526 

492 

t  Invitations  sent  in  September  2013 

Table  18.  Patient  Authorizations  and  Correlations. 


The  difference  in  the  number  of  veterans  that  responded  and  are  correlated  is  not  due  to 
patient  discovery  failures  but  rather  issues  with  VLER  authorizations  (e.g.  not  signed/dated,  not 
completed)  and  the  time  delay  for  both  organizations  to  "Opt-ln"  patients  into  their  system.  A 
small  percentage  of  invitations  have  been  "returned  to  sender"  for  various  reasons,  including 
those  regarding  deceased  patients  and  address  changes. 

John  Hargreaves,  Project  Manager,  met  with  three  physician  practices  to  discuss  VLER  and 
health  information  exchange,  including  two  independent  practices  (Dr.  Trostle/Dr.  Zeidan).  Dr. 
Zeidan  is  a  specialist  and  does  not  have  shared  patients  identified  in  the  list. 

eHealth  Exchange  Participant  Testing 

In  order  to  be  fully  compliant  with  eHealth  Exchange  requirements  after  implementing  a 
CONNECT  3. 3. 1.3  gateway,  Conemaugh  participated  in  the  new  eHealth  Exchange  testing 
program.  After  working  six  weeks  with  the  testing  body  (The  Certification  Commission  for 
Health  Information  Technology®)  and  support  contractor  for  the  Developers  Integration  Lab 
(Aegis),  Conemaugh  was  the  first  participant  in  the  nation  to  pass  the  2010  Smoke  Test  Cases, 
2011  Smoke  Test  Cases,  and  2011  Security  Test  Cases  on  September  4,  2013  (see  Appendix  6). 
Conemaugh  was  also  the  first  VLER  partner  to  complete  the  transition  to  2011  services. 
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CAL  and  Document  Assembler  Code  Donation 


In  partnership  with  NGC,  Conemaugh  completed  a  code  donation  to  the  FHA/CONNECT  team 
on  October  11,  2012  and  November  20,  2012.  The  second  donation  was  requested  in  order  to 
reduce  the  number  of  CheckStyle  and  PMD  violations  in  order  for  the  build  to  complete 
successfully.  As  of  the  end  of  the  period,  the  CONNECT  team  has  not  integrated  the 
Conemaugh  code  donations  into  the  source  code  tree.  Despite  this  delay,  the  binary  files  and 
instructions  were  made  available  to  the  community  on  October  18,  2012,  at  the  following  url: 

https://developer.connectopensource.org/displav/CONNECTR33/Dynamic+Document+Generat 

ion 

The  code  donations  were  also  reviewed  by  John  Hargreaves  and  Allen  Barger  during  the 
CONNECT  Code-A-Thon  on  November  7-8,  2012. 

Subcontractor  Transition 

Conemaugh's  subcontract  with  Northrop  Grumman  will  expire  on  September  30,  2013. 
Conemaugh  has  competitively  selected  CGI  Federal,  Inc.  as  the  new  vendor  to  support 
Conemaugh's  eHealth  Exchange  production  environment  on  an  as  needed  basis.  We  also 
received  proposals  from  Northrop  Grumman,  Cogon  Systems  and  MEDfx.  The  time  and 
material  agreement  is  being  drafted  and  should  be  executed  the  first  week  of  October. 

Before  Northrop  Grumman's  contract  expired,  MIDHT  equipment  was  moved  to  the  hospital 
data  center  in  Johnstown,  PA  on  September  23,  2013.  A  minimal  two-day  downtime  occurred 
while  the  servers  were  put  into  operation  in  the  new  location  with  appropriate  networking  gear 
and  Internet  service.  Follow-up  testing  was  completed  to  ensure  the  production  UDDI  change 
was  successful. 

Allscripts  Clinical  Analytics 

The  Allscripts  Clinical  Analytics  application  was  remotely  demonstrated  for  MIDHT  team,  CPG, 
and  MIS  staff  on  September  23,  2013.  All  MIDHT-related  questions  raised  were  addressed 
during  the  demonstration  or  via  follow-up  email  communications  from  Allscripts.  Current  plan 
is  to  contract  in  October  and  install  in  early  February  2014  as  soon  as  Allscripts  vll.4.1  upgrade 
is  operational. 
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Northrop  Grumman  Subcontractor  Support 

The  following  activities  are  selected  highlights  completed  by  Northrop  Grumman  during  the 
year. 

Cycle  4  development: 

Increased  the  max_allowed_packet  size  in  MySQL  to  allow  for  large  BLOBs  to  be  saved 
to  the  audit  repository  table. 

Modified  the  VA  style  sheet  so  that  the  date  values  in  the  Lab  Results  section  are 
formatted  and  the  header  of  each  C32  section  contains  information  about  the  patient 
(name,  date  of  birth). 

Moved  the  location  of  the  Lab  Results  comments  in  the  VA  style  sheet  from  the  last 
column  in  the  Lab  Results  table  to  a  new  row  that  directly  follows  the  current  entry  in 
order  to  reduce  document  size. 

Added  functionality  to  the  VA  style  sheet  that  allows  the  user  to  dynamically  change  the 
font  size  of  the  C32  content. 

Monitored  patient  correlations  and  communicated  updates  to  Conemaugh 
Generated  bi-weekly  audit  logs  and  delivered  to  Conemaugh 

Appended  "(INCLUDES  LAB  RESULTS)"  to  the  end  of  the  "34133-9  Summarization  of 
Episode  Note"  entry  in  the  CONNECT  Universal  Client  GUI's  Select  Document  Types 
drop-down  box. 

Determined  that  a  display  issue  with  the  Problem  date  values  was  the  result  of  the  XML 
that  is  returned  from  the  VA. 

On  January  17,  2013,  the  new  ENTRUST  Certificate  (valid  for  one  year)  was  successfully 
installed 

Hid  the  CONNECT  Universal  Client  GUI's  service  time  query  components  (VA  currently 
experiencing  an  issue  with  proper  handling)  and  added  the  Document  Query  response's 
serviceStartTime  value  to  the  Document  Query  results  table 

No  longer  populating  the  CHS  C32  Allergy  Event  Type  with  the  CAL  response  value. 
Instead,  set  the  Allergy  Event  Type  to  the  valid  Snomed  Code  value  of  "Propensity  to 
adverse  reactions  (disorder)" 

Inspected  the  Sig,  Order  Expiration,  Provider,  and  Dispense  Date  fields  in  the 
Medications  section  of  the  CHS  C32  for  missing  values 

Determined  the  CAL  Medications  response  does  not  contain  pharmacy  information 

Made  the  following  changes  in  production  to  the  Medications  section  of  the  Conemaugh 
C32  document  per  request  from  VA: 

o  Populated  the  substanceAdministration  text  tags  with  only  the  medication  Sig  value 
o  Put  the  section  text  <content>  tags  in  the  same  order  as  the  corresponding 
SubstanceAdministration  entries 
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o  Changed  the  ID  attribute  of  each  section  <content>  tag  to  be  the  medication  Sig  value 
instead  of  the  name  of  the  medication 

-  Always  display  "N/A"  for  service  time  in  the  Document  Search  Results  table  for  C32 
documents  that  are  returned  to  the  CONNECT  Universal  Client  GUI 

-  Corrected  the  spelling  of  what's  supposed  to  be  the  word  "Document"  in  the  first 
column  of  the  Document  Search  Results  table. 

-  Created  user  accounts  in  OpenDS  and  updated  passwords  as  requested. 

-  When  requested,  queried  the  MIDHT  Audit  Repository  to  confirm  the  Retrieve 
Document  actions  of  certain  VA  users. 

-  In  response  to  the  Healtheway  eHealth  Exchange  support  staff  assuming  administrative 
control  for  the  UDDI  domain  registrar  services,  connectivity  from  the  production 
Conemaugh  gateway  to  the  production  eHealth  Exchange  UDDI  was  tested  on  July  8, 
2013. 

-  Prevented  the  CONNECT  Universal  Client  GUI's  Logout  button  and  User's  Manual  link 
from  disappearing  when  a  user  selected  a  date  from  the  Patient  Search  tab's  calendar 
control. 

-  Provided  Conemaugh  documentation  regarding  how  to  efficiently  create  audit  log  files 
and  the  correlations  tables  in  Excel  format. 

-  Provided  Conemaugh  a  list  of  all  the  "projects"  that  have  been  changed  or  added  to 
support  Lab  Results. 

-  Provided  Conemaugh  with  documentation  regarding  how  new  MIDHT  users  are  added 
to  OpenDS. 

-  Provided  Conemaugh  with  instructions  on  how  to  determine  which  Conemaugh 
documents  are  provided  to  the  VA. 

-  Helped  Conemaugh  troubleshoot  an  issue  with  their  updated  UDDI  registry  entries. 


TATRC  Partners  and  Open  Source  Community  Outreach 

NGC  became  a  regular  participant  in  the  eHealth  Exchange  Spec  Factory  all-hands 
weekly  teleconference. 

NGC  continued  to  participate  in  the  bi-weekly  CONNECT  Sprint  Review  and  Planning 
Session  meetings. 

COGON  Systems  reached  out  to  Allen  Barger  for  help  with  validating  the  C62  document 
that  they  had  been  sending  to  the  VA.  Allen  helped  COGON  validate  their  C62 
document  by  performing  the  following  actions: 

o  Compared  the  COGON  C62  document  to  a  valid  Conemaugh  C62  document  and 
suggested  revisions 
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o  Loaded  the  COGON  C62  document  into  the  CAL  emulator,  and  then  rendered  it 
using  the  CONNECT  Universal  Client  GUI. 

On  July  1,  2013,  Allen  Barger  met  with  staff  members  from  Regenstrief  and  provided 
them  with  guidance  and  lessons  learned  regarding  C62  document  development. 

On  July  22,  2013,  Allen  Barger  participated  in  the  Healtheway  Testing  Program  Update. 

On  July  25,  2013,  Allen  Barger  dialed-in  to  the  Healtheway  Testing  Brief. 

At  the  suggestion  of  Conemaugh,  Allen  Barger  reached  out  to  members  of  FHA  to 
discuss  the  default  Availability  Status  value  in  CONNECT. 


Transition  to  CONNECT  v3.3.1.3 

Created  and  distributed  a  transition  plan  that  documents  the  tasks  that  are  required  to 
move  the  MIDHT  code  base  from  CONNECT  v3.1  to  CONNECT  v3.3.x. 

Created  a  Subversion  trunk  to  hold  version  documentation  that  is  pertinent  to  the 
installation  &  configuration  of  the  MIDHT  code  on  CONNECT  v3.3.1.3. 

Evaluated  versions  of  OpenDS  that  are  suitable  for  deployment  with  the  Java  install  that 
is  required  by  CONNECT  V3.3.1.3. 

Updated  the  MIDHT  code  base  to  work  with  the  CONNECT  v. 3. 3. 1.3  Core  Libraries. 

Provided  TATRC  the  CONNECTUniversalClientGUI  binaries  for  use  with  CONNECT 
V3.3.1.3 

Created  and  deployed  a  common  Java  library  for  the  CONNECT  Universal  Client  GUI  and 
the  Consumer  Preferences  Profile  GUI.  The  library  eliminates  the  need  to  have  the  same 
files  stored  in  both  the  CONNECT  Universal  Client  GUI  and  Consumer  Preferences  Profile 
web  applications. 

-  Created  and  deployed  an  updated  Initiate  adapter  for  use  with  CONNECT  v3.3.  The 
updated  adapter  eliminates  an  extra  web  service  call  and  facilitates  the  use  of  the 
CONNECT  build  script  to  compile  the  adapter's  code. 

Updated  the  AdpaterCommonDataLayerEJB,  AdapterDocumentAssemblyProxyEJB, 
DocumentAssemblyManagerEJB,  DocumentManagerEJB,  NHINAdapterServicesEJB,  and 
TemplateManagerEJB  projects  to  accommodate  the  CONNECTCoreLib  changes  in 
CONNECT  v3. 3. 1.3. 

Resolved  the  CONNECT  Universal  Client  GUI  issue  that  caused  patient  id's  to  be 
persisted  across  GUI  sessions  and  included  in  the  SAML  header  of  future  transactions. 

Sample  C32  and  C62  documents  were  produced  from  the  transitioned  MIDHT  code  and 
sent  to  Conemaugh  for  review. 
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The  CONNECT  v3.3.1.3  Entity  Doc  Query,  Adapter  Doc  Query,  and  NwHIN  Doc  Query 
requests  and  responses  were  compared  to  their  CONNECT  v3.1  counterparts. 

The  CONNECT  v3.3.1.3  C32  and  C62  documents  were  compared  to  their  CONNECT  v3.1 
counterparts. 

The  CONNECT  v3.3.1.3  document  repository,  audit  repository,  and  patient  correlation 
tables  were  compared  to  their  CONNECT  v3.1  counterparts. 

Work  began  on  transitioning  the  CONNECT  v3.3.1.3  code  from  Allen  Barger's 
development  machine  to  the  MIDHT-PREPROD33  server. 

A  CONNECT  bug  may  be  responsible  for  wrongly  populating  the  Issuer  and  Issuer  Format 
SAM L  Assertion  values  in  the  Audit  Log  and  Adapter  Document  Query  web  service 
requests. 

Work  was  completed  on  creating  a  document  that  outlines  how  to  install  and  configure 
the  MIDHT  code  stack,  built  from  CONNECT  v3.3.1.3,  on  a  Windows-based  server. 

The  following  changes  were  made  to  the  MIDHT  CONNECT  v3.3.1.3  code  base: 

o  Properly  formatted  the  Date  of  Birth  (DOB)  and  telephone  number  values  in  the 
Patient  Discovery  request 

o  Added  an  "XDSDocumentEntryType"  slot,  used  to  request  both  stable  and  on- 
demand  documents  from  eHealth  Exchange  participants,  to  the  Document  Query 
request 

o  Changed  the  value  of  the  "returnComposedObjects"  attribute,  located  in  the 
Document  Query  request,  from  "false"  to  "true" 

o  Changed  the  Issuer  SAML  header  value  to  be  compliant  with  the  NwHIN 
Authorization  Framework  and  Oasis  SAML  specifications. 

Delivered  a  document  to  Conemaugh  that  outlines  how  to  install  and  configure  the 
MIDHT  code  stack,  built  from  CONNECT  v3.3.1.3,  on  a  Windows-based  server. 

The  following  changes  were  made  to  the  MIDHT  CONNECT  v3.3.1.3  code  base: 

o  Set  the  Initiate  min  score  to  99  for  both  internal  and  external  patient  lookups 

o  Prevented  the  CONNECT  Universal  Client  GUI  from  caching  the  internal  Initiate 
min  score  after  a  patient  lookup  is  performed  in  the  GUI 

o  For  every  MIDHT  document  that  is  saved  to  the  document  repository,  populate 
the  document's  AvailabilityStatus  repository  field  with  "Available"  instead  of 
"Active" 

o  For  DQ  requests  that  originate  from  the  CONNECT  Universal  Client  GUI,  include  a 
$XDSDocumentEntryType  slot  in  the  request  if  the  targeted  community  supports 
the  2011  eHealth  Exchange  specifications. 
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o  To  more  easily  identify  incoming  2010  DR  requests,  the  print-lines  in  the  v2.0 
Document  Retrieve  Implementation  class  were  appended  with  a  2010 
designation 

o  Enabled  complete  builds  of  the  CONNECT  code  stack  by  updating  the 

AdapterFrameworklntTest  project  to  work  with  architectural  changes  that  were 
made  to  the  Dynamic  Document  Assembler  code 
o  Increased  the  Document  Query  web  service  Proxy  timeout  threshold. 

The  migration  of  the  CONNECT  v3.3.1.3  MIDHT  code  base  from  the  MIDHT-PREPROD33 
server  to  the  MIDHT-GOLIVE  server  was  completed  on  September  19,  2013. 


MIDHT  Data  Center  Configuration  and  Maintenance 

Conemaugh  granted  permission  to  NGC  to  relocate  the  MIDHT  equipment  to  a  data 
center  located  in  Chester,  VA.  The  NGC  facility  in  Johnstown,  PA  permanently  closed  on 
December  17,  2012. 

Essential  MIDHT  equipment  was  relocated  to  the  Chester,  VA  datacenter  during  the 
weekend  of  December  14,  2012. 


VA  Partner  and  eHealth  Exchange  Testing 

On  November  13,  2012,  VA  test  cases  were  executed  to  monitor  C32  changes  that  were 
made  in  support  of  VA's  Adapter  Maintenance  Release  5.0.1.  During  the  testing  process, 
several  Conemaugh  initiated  Patient  Discovery,  Document  Query  and  Document 
Retrieve  requests  were  sent  to  the  VA  test  system  and  the  returned  responses  were 
examined  for  accuracy. 

Northrop  Grumman  examined  the  VA  C32  module  content,  which  was  produced  during 
Adapter  5.0.1  validation  testing,  for  accuracy  and  reported  the  findings  to  Conemaugh. 

The  DIL  certificates  were  successfully  retrieved  from  Aegis  and  installed  on  the  MIDHT- 
PREPROD33  server. 

On  July  31,  2013,  Allen  Barger  attended  the  eHealth  Exchange  Participant  Testing 
Webinar. 

The  uddiConnectionlnfo_CONNECT33_41.xml  file,  which  is  used  to  direct  the  CHS 
gateway  to  the  DIL  endpoints,  was  successfully  installed  on  the  MIDHT-PREPROD33 
server. 

On  July  25,  2013,  Allen  Barger  met  with  John  Hargreaves  to  review  the  CONNECT  testing 
documentation  and  the  DIL  setup  and  administration  instructions. 

At  the  request  of  the  VA,  Document  Query  tests  were  conducted  for  test  patient 
NWHINONE  on  July  8,  15,  and  17,  2013.  The  test  results  were  extracted  from 
Conemaugh's  server  log  and  sent  to  the  VA  for  review. 
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Helped  determine  if  a  NullPointerException,  which  prevented  the  DIL  from  sending 
requests  to  the  Conemaugh  test  gateway,  was  originating  from  the  DIL  or  Conemaugh's 
gateway. 

On  August  1,  2013,  Allen  Barger  attended  a  Conemaugh  Test  Gateway  Kick-off  meeting, 
which  was  organized  by  the  VA. 

During  a  teleconference  on  August  13,  2013,  Allen  Barger  discussed  eHealth  Exchange 
testing  document  metadata  values  with  John  Hargreaves  from  Conemaugh  and  Heather 
Geier  from  CCHIT. 

Northrop  Grumman  helped  Conemaugh  interpret  a  DIL  error  message  regarding  a 
missing  patient  id,  and  determined  that  a  non-breaking  space  appended  to  the  end  of 
the  patient  id  was  the  cause  of  the  error. 

After  eHealth  Exchange  validation  testing  concluded,  the  certificates,  domain. xml  file, 
and  uddiConnectionlnfo.xml  file  on  the  MIDHT-PREPROD33  server  were  configured  for 
validation  testing  with  the  VA. 

On  September  11-12,  2013,  Allen  Barger  participated  in  the  validation  testing  sessions 
that  took  place  between  Conemaugh  and  the  VA. 


CONNECT  Code-a-Thon  Participation 

Allen  Barger  attended  the  CONNECT  Code-a-Thon,  held  at  Marymount  University  on 
November  7-8,  2012,  in  support  of  the  MIDHT  project. 

Allen  Barger  actively  participated  in  discussions  regarding  the  CONNECT  code  donation 
process  and  the  Dynamic  Document  Generation  plug-in  code  donated  by  Conemaugh. 

Lab  Result  Integration 

On  February  13,  2013,  Allen  Barger  and  Emily  Reynolds  attended  a  meeting  with 
Conemaugh  to  discuss  the  addition  of  Lab  Results  to  the  MIDHT  C32  document. 

-  Using  sample  messages  provided  by  Conemaugh,  NG  successfully  parsed  HL7  v2.2  Lab 
Result  messages  using  the  open-source  HL7  Application  Programming  Interface  (HAPI) 
library  to  HLV  v3. 

-  Assisted  Conemaugh  with  identifying  the  data  objects  that  comprise  a  HL7  v2.2 
message. 

Coding  efforts  have  produced  a  C32  document  containing  a  Lab  Results  section  that  is 
populated  with  static  Laboratory  Observation  data.  As  changes  are  made  to  the  Lab 
Results  section  of  the  C32  document,  the  document's  adherence  to  the  CDA/CCD-based 
specifications  are  routinely  validated  using  the  NIST  testing  web  site. 

-  In  order  to  determine  if  the  Lab  Results  Integration  code  needed  to  be  modified  to 
handle  multiple  Lab  Orders  and  Results  messages,  the  Northrop  Grumman  MIDHT  team 
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requested  additional  sample  Lab  messages  from  Conemaugh.  The  sample  messages 
were  received  by  the  Northrop  Grumman  MIDHT  team  on  April  8,  2013. 

In  anticipation  of  the  Conemaugh  Labs  interface  requiring  HL7  ORM  message  based 
requests,  Northrop  Grumman  and  Conemaugh  researched  the  content  and  structure  of 
the  ORM  message  type.  However,  it  has  since  been  determined  that  the  Conemaugh 
interface  will  require  ORU  message  based  requests. 

The  Document  Assembler  code  was  successfully  modified  to  accommodate  the  parsing 
and  interpretation  of  messages  containing  multiple  Lab  Orders  and  multiple  Lab  Results 
(for  a  given  Lab  Order). 

The  CHS  style  sheet  was  augmented  with  a  Lab  Results  section,  which  contains  a 
Date/Time  field,  a  Result  Type  field,  a  Result  Value  field,  a  Reference  Range  field,  an 
Interpretation  field,  a  Result  Status  field,  and  a  Source  field.  A  revised  CHS  C32 
document  containing  Lab  Results  was  delivered  to  Conemaugh  on  April  3,  2013  for 
review. 

Code  was  implemented  to  allow  the  represented  organization  for  each  Lab  result  to 
dynamically  change  based  on  the  hospital  code  value  in  the  MSH  section  of  the 
ORU_R01  message. 


Subtask  2.2  Provide  technical  and  documentation  assistance  on  DoD-managed  Virtual 
Lifetime  Electronic  Record  (VLER)  efforts. 


******No  Specjfjc  tasking  has  been  identified  by  TATRC****** 


Subtask  2.3  Investigate  productizing  a  Patient  Consent  module  using  established  standards, 
such  as  TP20/XACML. 

Conemaugh  and  NGC  agreed  to  participate  in  the  Jericho/UT-Austin  DS4P  Pilot  in  July  2012  by 
serving  as  a  test  CONNECT  gateway  on  a  simulated  eHealth  Exchange.  Due  to  contracts  and 
funding  issues  with  others,  the  pilot  began  work  in  April  2013.  More  information  can  be  found 
here  about  the  pilot:  http://wiki.siframework.org/Data+Segmentation+for+Privacy+Homepage 

This  pilot  explores  12  electronic  exchange  scenarios  and  eight  types  of  data  transactions.  To 
test  these  scenarios,  pilot  participants  perform  various  roles.  Jericho  Systems  serves  as  primary 
custodian  of  the  patient's  record  and  houses  the  Patient  Consent  Directive  (PCD)  repository. 
The  University  of  Texas  at  Austin  HIT  Program  simulates  the  role  of  a  research  university  that 
requests  the  patient's  record  and  subsequently  acts  as  the  secondary  custodian  of  the 
information.  Conemaugh  plays  the  role  of  a  marketing  network  that  becomes  the  second 
requestor. 
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In  support  of  the  DS4P  J-UT  pilot,  a  CentOS  Virtual  Machine  (VM)  image  was  successfully 
retrieved  from  Jericho  and  installed  on  a  MIDHT  test  sever  in  August  2013.  Once  installed,  the 
GlassFish  server  contained  within  the  VM  was  configured  to  use  ports  443  and  80  instead  of 
ports  8181  and  8080. 

On  September  19,  2013,  Northrop  Grumman  helped  the  Jericho  technical  team  establish  a 
GoTo  Meeting  session  that  was  used  to  support  the  Jericho/UT-Austin  Pilot  use-case 
demonstration.  Northrop  Grumman  will  provide  support  to  Jericho  until  their  contract  expires. 


Subtask  2.4  Assess  and  analyze  NHIN-related  activities,  to  include  data  center  performance 
metrics,  physician  evaluation  and  usage  of  the  NHIN  Portal,  and  resulting  benefits  of  HIE  with 
federal  participants. 

Utilization  Metrics 


All  organizational  and  IRB  approvals  were  received  during  the  previous  year.  On  July  16,  2013, 
the  Conemaugh  IRB  approved  the  continuation  of  the  study  for  an  additional  year.  On  July  24, 
2013,  the  U.S.  Army  Medical  Research  and  Materiel  Command's  (USAMRMC)  Office  of  Research 
Protections  (ORP),  Human  Research  Protection  Office  (ORP)  acknowledged  receipt  of  the 
continuing  review  documents  and  placed  them  in  the  protocol  file. 

Data  collection  continues  with  documentation  of  monthly  transactions  between  Conemaugh 
and  VA  for  treatment  purposes  as  reported  in  Figures  20,  21  and  22  below.  In  addition,  the 
number  of  sent  invitations,  returned  authorizations  and  successful  patient  discovery 
correlations  are  documented  in  Table  18  and  Figure  20  below.  Northrop  Grumman  has 
provided  monitoring  statistics  from  the  Glassfish  server  for  much  of  the  past  year  except  for  a 
two  month  period  because  the  server  unexpectedly  stopped  responding  on  July  23rd.  The 
review  and  analysis  of  data  center  performance  metrics  is  pending. 
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eHealth  Exchange 
Conemaugh/Altoona  VA 
Patient  Correlations  =  481 
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Oct-Dec  2012  Jan-Mar  2013  Apr-June  2013  July  -  Sept  2013 


Figure  20.  Patient  correlations  during  year. 


The  increase  in  patient  correlations  has  been  positive  and  has  been  mainly  dependent  on 
patient  response  to  the  VLER  packets  sent  via  postal  mail.  It  is  important  to  note  that  failed 
correlations  have  been  less  than  5%,  which  is  better  than  anecdotal  concerns  heard  in  other 
areas.  This  is  likely  due  to  the  inclusion  of  Social  Security  Number  used  to  make  a  match. 
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eHealth  Exchange 
Conemaugh/Altoona  VA 
140  Documents  Retrieved 


Oct-Dec2012  Jan-Mar2013  Apr-June2013  July- Sept  2013 


Figure  21.  Summary  Count  of  Clinical  Documents  Retrieved. 

The  increase  in  utilization  is  promising  for  the  future  of  the  eHealth  Exchange  and  will  likely 
increase  after  CMMC  completes  integration  of  lab  data  to  the  C32  document.  The  increase  in 
documents  retrieved  is  probably  due  to  the  flattening  of  the  learning  curve,  the  increase  in 
patient  correlations  and  the  increase  in  users  that  have  been  trained  on  the  new  system(s). 
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eHealth  Exchange 
Conemaugh/Altoona  VA 
Document  Type 

25  -r- 


CCD  O  CCD  I  ER  0  DS  0  RAD  O  RPT  I 


■  October  -  December  2012 

■  January  -  March  2013 

■  April  -  June  2013 

■  July  -  September  2013 


Figure  22.  Clinical  Documents  Retrieved  by  Type  and  Direction. 


CCD_I  =  Continuity  of  Care  Document  Inbound 
CCD_0  =  Continuity  of  Care  Document  Outbound 
RAD_0  =  Radiology  Report  Outbound 
DS_0  =  Discharge  Summary  Outbound 
ER_0  =  Emergency  Room  Note  Outbound 
RPT_I  =  C62  Reports  Inbound 


Conemaugh  providers  prefer  retrieving  VA  lab  results  and  medication  history  via  the  continuity 
of  care  document  (CCD).  Retrieval  of  C62  reports  from  the  VA  has  been  minimal  and  was  highly 
influenced  by  one  user/day.  Likewise,  VA  providers  have  preferred  to  retrieve  the  CCD  from 
Conemaugh  and  also  radiology  reports.  Retrieval  of  Emergency  Room  notes  and  discharge 
summaries  have  been  minimal  to  date.  Interestingly,  the  initiation  of  document  retrievals  is 
nearly  a  50/50  split  (Conemaugh  N  =  52,  VA  N=  48). 

A  total  of  22  male  veterans  have  responded  to  the  health  information  exchange  survey. 

Surveys  have  been  sent  to  37  veterans  for  whose  health  information  has  been  exchanged  in 
production.  Below  are  descriptive  results  from  a  few  important  questions: 
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Age: 
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Figure  23.  Survey  respondent  by  age. 


Gender: 


Figure  24.  Survey  respondent  by  gender. 
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Please  select  which  benefits  you  believe  were  achieved  as  a  result  of  HIE  between 
Conemaugh  and  Altoona  VA. 


20 


15 


10 
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0 


80.0  V. 


Improved  Patient  Safety  Less  Duplicate  Testing  I  am  Not  Sure 


Better  Coordination  Improved  Decision  Reduced  the  Need  to 

of  Care  Making/Quality  of  Care  Hand  Carry  Records 


Figure  25.  Patient  perception  of  HIE  benefits. 


Did  you  feel  your  health  information  was  secure  and  private? 


Figure  26.  Data  security  and  privacy. 
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The  limited  sample  size  of  Veterans  responding  to  the  survey  has  had  a  positive  experience  with 
health  information  exchange  between  Altoona  VA  and  Conemaugh.  The  majority  of  patients 
believe  that  coordination  of  care  across  providers  has  improved  and  has  reduced  the  need  for 
Veterans  to  hand  carry  their  paper  records  to  providers.  More  than  half  of  survey  respondents 
believe  that  decision  making  and  quality  of  care  has  also  improved  and  there  is  less  duplicate 
testing  because  of  the  electronic  exchange  of  medical  information.  Additional  surveys  will  be 
distributed  to  grow  the  sample  size. 

A  total  of  19  Conemaugh  Physician  Group  (CPG)  providers  have  responded  to  the  provider 
survey  regarding  health  information  exchange.  The  first  six  Conemaugh  practices  onboarded 
were  given  an  opportunity  to  participate  in  the  survey.  VA  has  not  provided  permission  yet  to 
survey  their  staff.  Please  note  only  system  users  (n=ll)  were  allowed  to  complete  all 
questions.  Below  are  results  for  a  few  important  questions: 


Please  select  your  position. 


Figure  27.  Survey  respondent  by  position. 
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Years  of  experience: 


Have  you  accessed  patient  health  information  through  the  NwHIN/VLER 

Exchange? 


Figure  29.  Provider  access  to  the  Exchange. 
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Please  indicate  which  documents  you  have  accessed  through  the  NwHIN/VLER 
Exchange?  (Please  select  all  that  apply) 


10- 


18.2  % 


9.1  %  9.1  %  9.1  *4  9.1  % 


Discharge  Summary  History  &  Physical  Mote  Radiology  Report 


Surgery  Report 


Continuity  of  Care 
Document  (indudes 
lab  results) 


ER  Reports 


Consultation  Note 


Procedure  Note 


Pathology  Report 


Figure  30.  Type  of  clinical  documents  accessed. 


I  am  more  satisfied  with  health  information  exchange  via  NwHIN  than  previous  routines 

(fax,  mail,  not  at  all). 


6' 


4 


2 


0 

Strongly  Agree  Moderately  Agree  Strongly  Disagree  Moderately  Disagree  Not  Applicable 


Figure  31.  Provider  satisfaction  with  HIE. 
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A  few  key  observations  are  noted  from  survey  results  and  personal  interaction  with  providers. 
Clerical  and  nursing  staffs  are  the  primary  users  requesting  data  via  the  eHealth  Exchange,  only 
one  CPG  physician  to  date  has  been  an  active  user.  System  usage  would  be  higher  but  in  some 
cases  Veterans  are  still  hand  carrying  their  records,  thus  reducing  the  need  to  request 
information  electronically.  The  most  requested  and  important  information  to  Conemaugh 
users  is  lab  results  performed  by  VA  facilities.  This  fact  promotes  continuity  of  care  and  a 
reduction  in  duplicate  testing.  As  aligned  with  patient  input,  providers  believe  coordination  of 
care  has  improved  and  patients  have  less  of  a  burden  on  them  to  hand  carry  paper  records. 

Not  surprisingly,  all  providers  believed  that  electronic  health  information  exchange  has  reduced 
the  number  of  requests  made  to  the  VA  medical  records  department.  Over  80%  of  providers 
agree  that  the  electronic  exchange  of  health  information  is  more  satisfying  than  previous 
routines  of  fax,  hard  copy  mail  or  not  having  the  information  at  all. 
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Key  Research  &  Development  Accomplishments 


Arm  1: 

•  Completion  of  data  collection  and  statistical  analysis  of  staff  and  student  nursing  satisfaction  of 
BCMA 

•  Completion  of  data  collection  and  statistical  analysis  of  pharmacy  and  nursing  workflow  analysis 
(time  and  motion) 

•  Intensive  review  of  CMMC  medication  errors  and  interim  statistical  analysis 

•  Implementation  of  second  MedCarousel  in  the  pharmacy 

•  Implementation  of  McKesson  BCMA  analytics 

Arm  2: 

•  Increasing  trend  in  patient  correlations,  system  users  and  document  exchanges 

•  Transitioned  to  CONNECT  3. 3. 1.3  gateway/adapter  in  production 

•  First  participant  in  nation  to  successfully  pass  new  "eHealth  Exchange"  testing  program 

•  One  year  in  production  phase  with  Altoona  VA  Medical  Center/VLER 


Reportable  Outcomes 

•  Submission  of  MIDHT  code  to  TATRC  and  Federal  Health  Architecture 

•  Public  presentations  at  CONNECT  Code-A-Thon  in  Maryland 


Conclusion 

The  MIDHT  project  continues  to  implement  and  research  health  information  technologies  (HIT) 
within  the  Conemaugh  Health  System,  located  in  Southwestern  Pennsylvania.  Core 
technologies  under  investigation  include  pharmacy  robotics,  bar  code  medication 
administration  (BCMA)  and  health  information  exchange  via  the  eHealth  Exchange.  Statement 
of  work  is  being  delivered  as  expected  with  few  technology  delays. 

Significant  progress  has  been  made  on  both  arms  of  the  project.  Research  activities  for  Arm  1 
have  progressed,  including  an  initial  analysis  on  medication  errors  and  completion  of  statistical 
testing  on  nursing  workflow  and  satisfaction.  CMMC  is  participating  in  the  14th  Virtual  Lifetime 
Electronic  Record  pilot  nationwide  via  data  exchange  with  the  Altoona  VA  Medical  Center  (Arm 
2).  Patient  correlations,  system  users  and  document  exchange  all  have  increased  throughout 
the  year.  CMMC  was  the  first  participant  in  the  nation  to  pass  the  new  "eHealth  Exchange" 
testing  program  (2010/2011  services). 
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Appendices 


Appendix  A- Acronym  List 

Acronym 

Description/Definition 

API 

Application  Programming  Interface 

BCMA 

Bar  Code  Medication  Administration 

CAL 

Common  Access  Layer 

CHS 

Conemaugh  Health  System 

CMMC 

Conemaugh  Memorial  Medical  Center 

COTS 

Commercial  Off  The  Shelf 

CPP 

Consumer  Preferences  Profile 

CVMH 

Conemaugh  Valley  Memorial  Hospital  (dba  "CMMC") 

DQ 

Document  Query 

DoD 

Department  of  Defense 

FHA 

Federal  Health  Architecture 

GUI 

Graphical  User  Interface 

HIE 

Health  Information  Exchange 

JIRA 

Bug  tracking  software 

LDAP 

Lightweight  Directory  Access  Protocol 

MHS 

Military  Health  System 

MIMC 

Conemaugh  Miners  Medical  Center 

MIDHT 

Military  Interoperable  Digital  Hospital  Testbed 

MIS 

Management  Information  Systems 

MPI 

Master  Patient  Index 

MYMC 

Conemaugh  Meyersdale  Medical  Center 

NGC 

Northrop  Grumman  Corporation 

NwHIN 

Nationwide  Health  Information  Network  (now  called  "eHealth 
Exchange") 

OID 

Object  Identifier 

ONC 

Office  of  the  National  Coordinator  for  HIT 

OpenDS 

Open  Directory  Service 

PD 

Patient  Discovery 

PHI 

Protected  Health  Information 

PMD 

A  static  ruleset  based  Java  source  code  analyzer  that 
identifies  potential  problems 

QD 

Query  for  Documents 

RCMR 

Message  for  document  metadata  and  content  query/response 

RD 

Retrieve  Documents 

SAML 

Security  Access  Markup  Language 

SAN 

Storage  Area  Network 

SOW 

Statement  of  Work 

57 


SRM 

Safety  and  Risk  Management 

SSO 

Single  Sign  On 

SVN 

Subversion 

TATRC 

Telemedicine  &  Advanced  Technology  Research  Center 

UC 

Universal  Client 

UDDI 

Universal  Description  Discovery  and  Integration 

USAMRMC 

United  States  Army  Medical  Research  and  Materiel  Command 

VA 

U.S.  Department  of  Veterans  Affairs 

VAMC 

Veterans  Affairs  Medical  Center 

VLER 

Virtual  Lifetime  Electronic  Record 

VNX 

Unified  Storage  Platform  by  EMC 

VPN 

Virtual  Private  Network 

WSDL 

Web  Services  Description  Language 

XDS 

Cross-Enterprise  Document  Sharing 
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Appendix  1  -  MedCarousel  2  Layout 
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Appendix  2  -  BCMA  Analytics  Reports 


McK  Adoption  Horizon  AdminRx  Analytics 


:ity:  01 


Refresh 


Department:  Show  All  I  -y 


Measure 

Previous  5  Months 

Current 

Month 

Goal 

Achievement 

Nov 

2012 

Dec 

2012 

Jan  2013 

Feb  2013 

Mar  2013 

Medication  Administration 

A 

Total  #  of  Med  Admin  Attempts 

X 

t 

315,216’ 

184,894’ 

156.625’ 

36,156’ 

%  Med  Admins  Given 

X 

t 

85  7%’ 

84.6%’ 

84.0%* 

83.8%* 

%  Med  Admins  w/Drug  Barcode  Scanned 

X 

t 

95,7%* 

95.0%’ 

94.3%’ 

94  5%* 

%  Med  Admins  Given  Early 

X 

t 

36%’ 

4.1%’ 

5.0%’ 

4  3%’ 

%  Med  Admins  Given  Late 

X 

t 

14.1%’ 

15.8%’ 

16.7%* 

16.7%’ 

%  Admins  wNo  Med  Order  Found  Override 

X 

t 

1.1%* 

1.2%’ 

1.3%’ 

1.3%’ 

%  Admins  w /  Quantity  Override 

X 

t 

0.7%’ 

0  7%’ 

0.7%’ 

06%’ 

%  Admins  w/No  Schedule  Found  Override 

X 

t 

1.2%’ 

1.3%’ 

1.4%’ 

14%’ 

- 

Medication  Administration  Warnings 

Total  #  of  Med  Admin  Warnings 

t 

t 

180,638’ 

126,397* 

112,451* 

25,483* 

%  Med  Admin  Warnings  Overridden 

t 

t 

44.6%’ 

48.3%’ 

48.9%’ 

47  3%’ 

%  Med  Admin  Warnings  Leading  to  Modify 

t 

t 

524%’ 

48.7%’ 

48  5%* 

49.8%’ 

%  Med  Admin  Warnings  Leading  to  Cancel 

t 

X 

52.4%’ 

48.7%’ 

48.5%’ 

49  8%’ 

1 

V  Administration 

Total  #  of  IV  Administrations 

t 

X 

13,345’ 

8,535* 

8,866’ 

1,910’ 

%  IV  Admin  Starts 

t 

X 

48.1%* 

47.8%’ 

48.3%’ 

49  3%’ 

%  IV  Starts  w /  IV  Barcode  Scanned 

t 

t 

71.3%’ 

63.3%’ 

66.2%’ 

70.5%’ 

%  IV  Admins  w/  Wristband  Scanned 

t 

t 

88  9%’ 

90.1%’ 

90  2%’ 

89.1%’ 
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McK  Adoption  HARx  Analytics  Medication  Administration 


Month  Medication 

was  Charted 

Total  Med 
Admin 
Attempts 

Given 

% 

Barcode 

Scanned 

Schedule 

Override 

Count 

%  wSchedule 
Override 

Drug 

Warning 

Override 

Count 

Quantity 

Override 

Count 

% 

wQuantity 

Override 

2013_01_January 

321,244 

275,332 

95.6% 

3,305 

1.2% 

3,067 

2,026 

0.7% 

2013_02_February 

188,333 

159.506 

95.0% 

2,005 

1.3% 

1,910 

1,078 

0.7% 

2013_03_March 

159,407 

133.921 

94.3% 

1,831 

1.4% 

1,689 

954 

0.7% 

2013_04_April 

16,363 

13,612 

94.8% 

208 

1.5% 

197 

69 

0.5% 

Report  Total 

685,347 

582,371 

95.1% 

7,349 

1.3% 

6,863 

4,127 

0.7% 

.  Top  of  page  . 

|lT  Adoption  Horizon  Admin  Rx  Analytics  1 .2 


.  Top  of  page  . 


Logout 


61 


rmance  Analytics1 


Dashboard 


Highlights 


Reports 


Resources 


Display 


Alerts 


Drill  Up 


Drill  Top 


Export 


Print 


Preferences 


Startup 


Scorecards 


V  Logout  I 


McK  Adoption  HARx  Analytics  Medication  Administration 


Drill  1  of  8:  Month  Medication  was  Charted 

>  Drill  2  of  8:  Primary  Medication  Name:  from  Month  Medication  was  Charted  /  2013_03_March 


Page  1  of8  [  1  2  3  4  5  6  7  8 


Primary  Medication  Name 

Total  Med 
Admin 
Attempts 

Given 

%  Given 

Barcode 

Scanned 

Count 

% 

Barcode 

Scanned 

Schedule 

Override 

Count 

%  wSchedule 
Override 

Drug 

Warning 

Override 

Count 

%  wDrug 
Warning 
Override 

Given 

Late 

% 

Given 

Late 

Given 

Early 

% 

Given 

Early 

METOPROLOL  TARTRATE 

2,904 

2,552 

87  9% 

2,332 

91  4% 

11 

0.4% 

11 

0.4% 

480 

18  8% 

49 

19% 

SODIUM  CHLORIDE  0.9  % 

2,899 

1.952 

67.3% 

1,720 

88.1% 

144 

7.4% 

146 

7.5% 

459 

23.5% 

7 

0.4% 

INSULIN  LISPRO 

2,779 

2,515 

90.5% 

2,344 

93.2% 

22 

0.9% 

15 

0.6% 

82 

3.3% 

6 

0.2% 

POTASSIUM  CHLORIDE 

1,808 

1,579 

87.3% 

1,547 

98  0% 

11 

0.7% 

11 

0.7% 

474 

30.0% 

22 

1.4% 

ENOXAPARIN  SODIUM 

1,745 

1,570 

90.0% 

1.543 

98.3% 

1 

0.1% 

1 

0.1% 

476 

30.3% 

57 

36% 

HEPARIN  SODIUM  (PORCINE) 

1,743 

1,578 

90.5% 

1,546 

98  0% 

10 

0.6% 

11 

0.7% 

213 

13.5% 

61 

3.9% 

LORAZEPAM 

1,664 

1.534 

92.2% 

1.460 

95.2% 

42 

2.7% 

37 

2.4% 

130 

8.5% 

68 

4  4% 

ACETAMINOPHEN 

1,639 

1.567 

95.6% 

1,526 

97.4% 

18 

1.1% 

12 

0.8% 

67 

4  3% 

12 

0.8% 

GABAPENTIN  CAP 

1,583 

1,492 

94.3% 

1,487 

99  7% 

0 

0.0% 

0 

0.0% 

218 

14.6% 

90 

6  0% 

SIMVASTATIN 

1,478 

1,411 

95.5% 

1,396 

98.9% 

1 

0.1% 

1 

0.1% 

153 

10  8% 

114 

8.1% 

FENTANYL  CITRATE  (PF) 

1,424 

1,404 

98.6% 

1,241 

88.4% 

91 

6.5% 

86 

6.1% 

4 

0.3% 

0 

0.0% 

OXYCODONE 

1,372 

1,317 

96.0% 

1,299 

98  6% 

2 

0.2% 

2 

0.2% 

86 

6.5% 

12 

0  9% 
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Drill  1  of  8:  Month  Medication  was  Charted 

Drill  2  of  8  Primary  Medication  Name:  from  Month  Medication  was  Charted  /  2013_03_March 
.  Drill  3  of  8  Department  where  Medication  was  Given  from  Primary  Medication  Name  /  LORAZEPAM 


Department 
where 
Medication 
was  Given 

Total  Med 

Given 

% 

Given 

Barcode 

Scanned 

Count 

% 

Barcode 

Scanned 

Schedule 

Override 

Count 

%  wSchedule 
Override 

Drug 

Warning 

Override 

Count 

%  wDrug 
Warning 
Override 

Given 

Late 

% 

Given 

Late 

Given 

Early 

% 

Given 

Early 

Admin 

Attempts 

G4 

184 

170 

924% 

156 

91.8% 

4 

2.4% 

4 

2.4% 

11 

6  5% 

5 

2.9% 

G7 

167 

154 

92.2% 

130 

84.4% 

2 

13% 

2 

1.3% 

29 

18  8% 

2 

1.3% 

R1 

128 

95 

74.2% 

90 

94  7% 

0 

0.0% 

0 

0.0% 

5 

5.3% 

3 

3.2% 

A1 

123 

113 

91  9% 

111 

98  2% 

1 

0.9% 

1 

0.9% 

9 

8.0% 

6 

5.3% 

GS 

101 

96 

95.0% 

91 

94.8% 

2 

2.1% 

2 

2.1% 

8 

8.3% 

2 

2.1% 

G6 

97 

94 

96  9% 

92 

97.9% 

2 

2.1% 

2 

2.1% 

12 

12.8% 

3 

3.2% 

A8 

92 

87 

94.6% 

86 

98  9% 

1 

1.1% 

1 

1.1% 

7 

8.0% 

1 

11% 

E4 

73 

71 

97.3% 

69 

97.2% 

1 

1.4% 

0 

0.0% 

7 

9.9% 

2 

2.8% 

A6 

68 

64 

94.1% 

61 

953% 

4 

6.3% 

4 

6  3% 

1 

1.6% 

1 

1.6% 

R8 

64 

55 

85.9% 

54 

98.2% 

1 

1.8% 

1 

1.8% 

2 

3.6% 

3 

5.5% 

R9 

59 

58 

98  3% 

52 

89.7% 

0 

0.0% 

0 

0.0% 

3 

5.2% 

1 

17% 

E6 

50 

48 

96.0% 

47 

97.9% 

2 

4.2% 

1 

2.1% 

2 

4.2% 

0 

00% 

M7 

44 

39 

88.6% 

38 

97  4% 

0 

0.0% 

0 

0.0% 

3 

7.7% 

0 

0.0% 

7R 

39 

35 

89.7% 

33 

94.3% 

15 

42.9% 

13 

37.1% 

2 

5.7% 

0 

0.0% 

A9 

28 

25 

89.3% 

22 

88  0% 

0 

0.0% 

0 

0.0% 

0 

0  0% 

0 

0.0% 

MF 

18 

17 

94.4% 

16 

94.1% 

0 

0.0% 

0 

0.0% 

0 

0.0% 

0 

0.0% 

S5 

7 

6 

85.7% 

5 

83.3% 

0 

0.0% 

0 

0.0% 

0 

0.0% 

0 

0.0% 

Total 

1,342 

1,227 

91.4% 

1,153 

94.0% 

35 

2.9% 

31 

2.5% 

101 

8.2% 

29 

2.4% 
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Medication  Administration  System  Survey  -  Nurses’  Assessment  Survey 

1.  Welcome 


Thank  you  tor  participating  in  the  Medication  Administration  System  Survey  - 
Nurses  Assessment  Survey  (MAS-NA5)  project 

Completing  this  form  indicates  you  fecogmze  that  this  is  a  research  project.  As  a  token  of  our  appreciation,  you  wilt  receive  a  $20.00  incentive  from 
Crossroads  cafe  or  similar  venue  when  you  have  completed  the  survey  After  completion  please  call  814*269-5232  to  arrange  receipt  and  please 
have  your  “SUM"  available  We  reserve  the  right  to  withhold  incentive  for  any  reason  related  but  not  limited  to  that  which  compromises  the  survey 
intent,  data  validity,  completion  compliance,  or  respondent  eligibility. 

If  you  previously  completed  a  MAS-NAS  please  complete  this  survey  so  we  can  learn  how  barcodefeMAR  has  changed  how  you  administer 
medications  We  will  compare  nurses'  responses  on  this  MAS-NAS  with  previous  surveys  completed  In  2011  and  2012  to  learn  the  impact  of 
converting  to  barcodeleMAR  on  CHS  nurses. 

if  this  is  your  first  MAS-NAS,  we  would  like  to  learn  your  beliefs  about  the  system 

Please  create  a  five  digit  confidential  identifier  called  SUM  This  allows  us  to  maintain  your  privacy  while  being  able  to  compare  surveys  taken  at 
different  times.  Even  if  this  is  your  first  survey,  please  compute  your  SUM.  We  ask  you  to  calculate  the  following  number  and  place  it  in  the  space 
provided.  The  number  is  the  SUM  of  the  last  five  digits  of  your  social  security  number  added  to  the  last  five  digits  of  your  home  telephone  number 
If  your  telephone  number  changed,  please  use  the  number  you  previously  used.  Only  the  SUM  of  those  two  numbers  should  be  recorded  on  the 
first  page  of  the  survey.  If  your  sum  has  more  than  five  digits,  record  only  the  last  five  See  example  below 

Last  five  social  secunty  numbers  =  52346 

4- 

Last  five  home  telephone  numbers  =  58721 
SUM  =  111067 

ENTER  1 1067  Mast  five) 

After  calculating  your  SUM,  please 

Add  your  SUM  on  the  next  page  and  complete  the  survey 

Thank  you  for  your  participation  in  this  important  project* 


2.  Introduction 


*  1.  SUM: 


*2.  Date  of  completing  the  survey: 


Pnyv  > 
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Medication  Administration  System  Survey  -  Nurses’  Assessment  Survey 


*3.  Nursing  Unit: 

Meyersdale  Medical  Center 
Miners  Medical  Center 
Ashman  6  ICU 
Ashman  7 
Ashman  8 
Ashman  9 
Ashman  10 
Rose  6  ICU 
Rose  7  CICU 
Rose  7  Telemetry 
Rose  8 
Rose  8  PCU 
Rose  9 
Rose  10 
E4 
E0 
GS  4 
GS5 
GS  6 
ASU 
PACU 

Crichton  Rehab 
Med  Surg  7 
RICN 
Maternity 
Pediatrics 
Geropysch 
Aloysia  Hall 
Behavioral  Health 
School  of  Nursing 

I 


Float 


Medication  Administration  System  Survey  -  Nurses’  Assessment  Survey 


Otcu 

Other  (please  specify) 


The  following  statements  describe  experiences  and  beliefs  about  the  current  medication  administration  system  you  use  By  current  medication 
administration  system,  we  mean  the  institutional  structures  and  operational  policies  that  direct  and  support  the  process  and  procedures  of 
delivering/administering  pharmaceutical  products  to  patients.  This  includes  all  activities  from  medication  ordering  to  dispensing  to  administration 
and  documentation  as  well  as  all  supplies  and  technologies  associated  with  the  current  medication  system,  such  as  computer  terminals,  medication 
storage  units,  drug/drug  interaction  information,  patient  education  sheets,  infusion  pumps,  tubings,  syringes,  etc  that  are  required  in  the  process  of 
getting  medications  to  patients. 

Completing  this  survey  should  take  about  10  minutes 

By  completing  this  survey,  you  are  indicating  consent  to  participate  in  this  evaluation. 

Please  read  each  statement  and  circle  the  number  that  best  expresses  your  own  experiences  and  beliefs 
There  are  no  right  or  wrong  answers. 

Do  not  write  your  name  on  the  survey. 

Please  indicate  the  degree  to  which  you  agree  or  disagree  with  each  item  by  using  the  following  RATING  SCALE 
Choose: 

1  if  you  strongly  agree  with  the  statement. 

2  if  you  moderately  agree  with  the  statement. 

3  if  you  slightly  agree  with  the  statement 

4  if  you  slightly  disagree  with  the  statement. 

5  if  you  moderately  disagree  with  the  statement 

6  if  you  strongly  disagree  with  the  statement. 

NA  if  the  statement  does  not  apply  to  you. 

Example:  The  current  medication  administration  system  provides  adequate  information  about  possible  mediation  side  effects. 

Answer:  If  you  believe  that  you  always  have  access  to  the  type  and  amount  of  information  you  need  to  be  able  to  assess  for  and  manage  potential 
side  effects  of  medications  you  administer,  you  should  circle  1  because  that  statement  best  expresses  your  belief 


3.  Questions 


4.  Because  of  information  available  through  the 
current  medication  administration  system  I  know  both 
the  intended  actions  and  side  effects  of  medications  I 
administer. 


O’ 

O2 

O’ 

o< 

O’ 

O’ 

o 

z 

> 

Strongly 

Moderately 

Slightly 

Slightly 

Moderately 

Strongly 

Not 

Agree 

Agree 

Agree 

Disagree 

Disagree 

Disagree 

Applicable 

5. 1  find  the  drug  alert  feature  (drug/drug  or  drug/food 
interaction)  of  the  current  medication  administration 
system  helpful. 

O1  O2  O  O4  O5  O6  Ona 

Strongly  Moderately  Slightly  Slightly  Moderately  Strongly  Not 

Agree  Agree  Agree  Disagree  Disagree  Disagree  Applicable 
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Medication  Administration  System  Survey  -  Nurses’  Assessment  Survey 


6.  The  current  medication  administration  system 
makes  it  easy  to  check  active  medication  orders 
before  administering  medications. 

O1  O2  O3  O4  O5  O6  Ona 

Strongly  Moderately  Slightly  Slightly  Moderately  Strongly  Not 

Agree  Agree  Agree  Disagree  Disagree  Disagree  Applicable 

7.  The  current  medication  administration  system 
provides  me  with  information  to  know  that  a 
medication  order  has  been  checked  by  a  pharmacist 
before  I  administer  the  medication. 

O1  O2  0304  O5  O6  Ona 

Strongly  Moderately  Slightly  Slightly  Moderately  Strongly  Not 

Agree  Agree  Agree  Disagree  Disagree  Disagree  Applicable 

8.  The  current  medication  administration  system 
promotes  2-way  communication  between  clinicians 
(MD,  Pharmacist,  RN)  about  medication  orders. 

O1  O2  O3  O4  O5  Os  Ona 

Strongly  Moderately  Slightly  Slightly  Moderately  Strongly  Not 

Agree  Agree  Agree  Disagree  Disagree  Disagree  Applicable 


9. 1  have  access  to  the  systems  that  support 
medication  administration  (physician’s  orders,  drug 
information)  when  I  need  them. 

O1  O2  O  O4  O5  O®  Ona 

Strongly  Moderately  Slightly  Slightly  Moderately  Strongly  Not 

Agree  Agree  Agree  Disagree  Disagree  Disagree  Applicable 

10.  The  drug  information  available  through  the  current 
medication  administration  system  is  easy  to  get  when 
I  need  that  information. 

O1  O2  O  O4  O 5  O6  Ona 

Strongly  Moderately  Slightly  Slightly  Moderately  Strongly  Not 

Agree  Agree  Agree  Disagree  Disagree  Disagree  Applicable 
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11.1  know  where  all  the  medications  I  need  are  stored 
(either  on  the  unit  or  if  they  need  to  be  procured  from 
the  pharmacy). 

O1  O2  O3  O4  O5  O6  Ona 

Strongly  Moderately  Slightly  Slightly  Moderately  Strongly  Not 

Agree  Agree  Agree  Disagree  Disagree  Disagree  Applicable 

12.  The  current  medication  administration  system 
helps  me  to  be  efficient  at  medication  administration. 

O1  O2  O3  O4  O5  O8  Ona 

Strongly  Moderately  Slightly  Slightly  Moderately  Strongly  Not 

Agree  Agree  Agree  Disagree  Disagree  Disagree  Applicable 

1 3.  The  current  medication  administration  system 
makes  it  easy  to  check  that  I  am  following  the  “5 
rights”  when  I  administer  medications. 

O1  O2  0304  O5  O8  Ona 

Strongly  Moderately  Slightly  Slightly  Moderately  Strongly  Not 

Agree  Agree  Agree  Disagree  Disagree  Disagree  Applicable 

14.  The  turnaround  time  for  receiving  medications 
needed  “stat”  or  for  patients  newly  admitted  to  the 
unit  is  adequate. 

O1  O2  O3  O4  O5  O8  Ona 

Strongly  Moderately  Slightly  Slightly  Moderately  Strongly  Not 

Agree  Agree  Agree  Disagree  Disagree  Disagree  Applicable 

15.  The  current  medication  administration  system  is 
effective  in  reducing  and  preventing  medication 
errors. 

O1  O2  O3  O4  O5  O8  Ona 

Strongly  Moderately  Slightly  Slightly  Moderately  Strongly  Not 

Agree  Agree  Agree  Disagree  Disagree  Disagree  Applicable 

16.  The  current  medication  administration  system  is 
user-friendly  to  the  nurses  who  administer 
medications. 

□ 1  □ 2  D3  □"  □ 5  □  «  Dna 

Strongly  Moderately  Slightly  Slightly  Moderately  Strongly  Not 

Agree  Agree  Agree  Disagree  Disagree  Disagree  Applicable 
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17.  The  equipment  and/or  supplies  needed  to 
administer  medications  are  readily  available  to  me. 

O'  O 2  O3  O  O 5  O 6  Ona 

Strongly  Moderately  Slightly  Slightly  Moderately  Strongly  Not 

Agree  Agree  Agree  Otsagree  Disagree  Disagree  Applicable 

18.  Information  available  through  the  current 
medication  administration  system  helps  me  to  know 
what  to  do  should  my  patient  have  any  bad  reactions 
from  a  medication. 

O1  O2  O  O'  O5  O9  Ona 

Strongly  Moderately  Slightly  Slightly  Moderately  Strongly  Not 

Agree  Agree  Agree  Oisagree  Disagree  Disagree.  Applicable 

19. 1  have  to  keep  stashes  of  medications  to  be  sure  I 
have  medications  I  need  when  I  need  them. 

0 1  0 1  O 0 1  O 5  O  °  O  na 

Strongly  Moderately  Slightly  Slightly  Moderately  Strongly  Not 

Agree  Agree  Agree  Disagree  Disagree  Disagree  Applicable 


5.  Open-ended 


20.  Please  add  any  comments  you  wish  about  the  current  medication  administration 
system  and  the  degree  to  which  components  of  the  current  system  support  your  ability  to 
administer  medications  safely  and  professionally. 

I  ^ 

I _ J 

21.  if  you  could  change  one  thing  in  the  current  medication  administration  system,  what 
would  it  be? 

i  a 


6.  BCMA 


Priyi*  6 
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22.  Did  you  work  on  an  inpatient  nursing  unit  at  CHS  before  BCMA  ( Admin-Rx)  was 
implemented? 

O Yes  O'¬ 


Comparing  now  (using  barcode/eMAR)  with  the  old  system,  please  respond  to  the  following  7  statements. 

23.  It  is  easier  to  do  all  the  checking  steps  needed  during  the  medication  administration 

process. 

Strongly  Moderately 

Q  Slightly  Q  Slightly 

Moderately  Strongly  Did  not  use 

Agree  Agree 

Agree  Disagree 

Disagree  Disagree  old  system 

24.  This  is  a  safer  system  for  patients. 

Strongly  Agree 

Slightly  Disagree 

Did  not  use  old  system 

Moderately  Agree 

Q  Moderately  Disagree 

Slightly  Agree 

Strongly  Disagree 

25.  With  the  new  system,  it  is  easier  to  access  information  1  need  to  administer 

medications. 

Strongly  Agree 

Slightly  Disagree 

Did  not  use  old  system 

Moderately  Agree 

Moderately  Disagree 

Slightly  Agree 

Strongly  Disagree 

26. 1  am  more  satisfied  with  this  new  medication  administration  system  than  with  the 

previous  one. 

Strongly  Agree 

Slightly  Disagree 

Did  not  use  old  system 

Moderately  Agree 

Moderately  Disagree 

Slightly  Agree 

Strongly  Disagree 

27. 1  have  more  time  to  spend  with  patients. 

Strongly  Agree 

Slightly  Disagree 

Did  not  use  old  system 

Moderately  Agree 

Moderately  Disagree 

Slightly  Agree 

Strongly  Disagree 

Page  7 


Medication  Administration  System  Survey  -  Nurses’  Assessment  Survey 


28.  Barcode/eMAR  has  made  the  medication  administration  process  more  efficient  for  me. 

o  Strongly  Agree  o  Slightly  Disagree  o  Did  not  use  old  system 

Moderately  Agree  Moderately  Disagree 

o  Slightly  Agree  o  Strongly  Disagree 

29.  Medications  are  more  readily  available  when  I  need  them  for  patients. 

o  Strongly  Agree  o  Slightly  Disagree  o  Did  not  use  old  system 

o  Moderately  Agree  o  Moderately  Disagree 

o  Slightly  Agree  o  Strongly  Disagree 


8.  Closing 


*30.  Overall,  how  satisfied  are  you  with  the  current  medication  administration 
system?  Please  select. 

O1-  O2  O3  O4  O5-  O6  O7  O8  O9  O10 


Completely 

Dissatisfied 


Neither 

satisfied 

nor 

dissatisfied 


Completely 

Satisfied 


*31.  How  long  have  you  been  using  the  BCMA  system  (Admin-Rx)  with  patients? 

o  Less  than  1  month  o  4-6  months  o  10-12  months 

o  1-3  months  o  7-9  months 

Please  provide  the  following  information  about  yourself  and  your  background. 

32.  Gender: 

o  Male  O  Female 

33.  Age: 


34.  Highest  nursing  degree: 

o  LPN  o  Diploma 


O' 


o 


o 


35.  Number  of  years  employed  as  a  nurse: 


*36.  Number  of  years  employed  by  Conemaugh: 
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37.  Number  of  hours  worked  in  a  typical  week: 


38.  Typical  shift  rotation  schedule: 


o  All  Shifts 
o  Evenings 

Nights 

39.  Typical  weekly  schedule: 

o  mostly  weekend/holiday 
o  mostly  Monday-Friday 

*40.  Current  position: 

staff/direct  patient  care  (staff  RN/NIC) 
o  leadership/management 
Other  (please  specify) 


o 
o 

o  All  Days 


Rotate  D/E 
Rotate  D/N 


o 

o 


7A-7P 

7P-7A 


o  rotate  weekdays/weekends/holidays 


o 

o 


education 

Student 


41.  Compared  to  your  nursing  peers,  how  do  you  rate  your  computer  skills? 

Above  Average  Average  (^)  Below  Average 

42.  Do  you  use  a  computer  at  home? 

O Yes  o 


43.  How  do  your  rate  your  skill  at  obtaining  patient  care  information  from  the  Conemaugh 
computer  systems? 

o  Excellent  o  Good  o  Fair  Q  Poor 


44.  Did  you  ever  use  barcode/eMAR  in  another  hospital  before  working  at  CHS? 

o  Yes  o  No 

Copyright  by  Ann  C.  Hurley.  Diane  R.  Lancaster.  Judy  Hayes.  Anne  Bane.  Chantel  Wilson  Chase.  2003.  Do  not  copy,  circulate,  or  cite  without 
written  permission  of  the  researchers. 
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Appendix  4  -  Nurse  Shadowing 


Memorial  Medical  Center  (CMMC) 


CMMC  PRE  (N  =  26  observations  -  RN): 


Nurse  Shadowing:  Study  Time  Period 

=  PRE, 

Location  =  MMC 

Processes  ranked  by  Total  Time  per  PID 

PID  ranked  by  Count 

PID 

cumulative 

time, 

hh:mm:ss 

cumulative 

%  of  time 

%  of 

Total  Time 
per  PID 

Total  Time 
/  PID, 

hh:m  m  :ss 

PID 

PID 

Count  of 
Activity 

IDs 

avg.  time  / 
AID, 

hh:mm:ss 

4 

20:48:13 

19.7% 

19.7% 

20:48:13 

4 

4 

1290 

0:00:58 

1 

37:50:29 

35.9% 

16.2% 

17:02:16 

1 

7 

1217 

0:00:33 

2 

53:28:41 

50.8% 

14.8% 

15:38:12 

2 

1 

987 

0:01:02 

3 

68:09:45 

64.7% 

13.9% 

14:41:04 

3 

9 

600 

0:00:50 

7 

79:14:10 

75.2% 

10.5% 

11:04:25 

7 

3 

559 

0:01:35 

9 

87:35:16 

83.1% 

7.9% 

8:21:06 

9 

2 

252 

0:03:43 

10 

92:57:44 

88.2% 

5.1% 

5:22:28 

10 

8 

227 

0:01:12 

8 

97:30:48 

92.5% 

4.3% 

4:33:04 

8 

10 

90 

0:03:35 

6 

98:30:09 

93.5% 

0.9% 

0:59:21 

6 

6 

51 

0:01:10 

5 

98:37:32 

93.6% 

0.1% 

0:07:23 

5 

5 

13 

0:00:34 

Table  19.  CMMC  PRE:  Details  by  Process. 


CMMC  POST  (N  =  26  observations  -  RN): 


Nurse  Shadowing:  Study  Time  Period  = 

POST, 

Location  =  MMC 

Processes  ranked  by  Total  Time  per  PID 

PID  ranked  by  Count 

PID 

cumulative 

time, 

hh:mm:ss 

cumulative 

%  of  time 

%  of 

Total  Time 
per  PID 

Total  Time 
/  PID, 

hh:m  m  :ss 

PID 

PID 

Count  of 
Activity 

IDs 

avg.  time  / 
AID, 

hh:mm:ss 

1 

22:10:21 

21.1% 

21.1% 

22:10:21 

1 

7 

1261 

0:00:33 

2 

41:42:02 

39.8% 

18.6% 

19:31:41 

2 

1 

1249 

0:01:04 

4 

60:07:50 

57.3% 

17.6% 

18:25:48 

4 

4 

1109 

0:01:00 

3 

72:12:12 

68.8% 

11.5% 

12:04:22 

3 

9 

598 

0:00:55 

7 

83:46:53 

79.9% 

11.0% 

11:34:41 

7 

3 

491 

0:01:29 

9 

92:56:34 

88.6% 

8.7% 

9:09:41 

9 

2 

363 

0:03:14 

8 

98:28:14 

93.9% 

5.3% 

5:31:40 

8 

8 

221 

0:01:30 

10 

100:57:21 

96.2% 

2.4% 

2:29:07 

10 

10 

39 

0:03:49 

6 

101:05:17 

96.4% 

0.1% 

0:07:56 

6 

6 

8 

0:00:59 

5 

101:07:23 

96.4% 

0.0% 

0:02:06 

5 

5 

3 

0:00:42 

Table  20.  CMMC  POST:  Details  by  Process. 
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Meyersdale  Medical  Center  (MYMC) 


MYMC  PRE  (N  =  1  observation  -  LPN): 


Nurse  Shadowing:  Study  Time  Period  = 

■  PRE,  Location  =  MYMC 

Processes  ranked  by  Total  Time  per  PID 

PID  ranked  by  Count 

PID 

cumulative 

time, 

hh:mm:ss 

cumulative 

%  of  time 

%  of  Total  Time 

Total  Time  /  PID, 

per  PID  hh:mm:ss 

PID 

PID 

Count  of 
Activity 

IDs 

avg.  time  / 
AID, 

hh:mm:ss 

10 

1:06:28 

28.3% 

28.3% 

1:06:28 

10 

1 

35 

0:00:55 

3 

1:53:05 

48.1% 

19.8% 

0:46:37 

3 

7 

17 

0:00:41 

4 

2:27:58 

63.0% 

14.8% 

0:34:53 

4 

4 

16 

0:02:11 

1 

3:00:04 

76.6% 

13.7% 

0:32:06 

1 

3 

7 

0:06:40 

7 

3:11:43 

81.6% 

5.0% 

0:11:39 

7 

10 

6 

0:11:05 

9 

3:20:48 

85.4% 

3.9% 

0:09:05 

9 

9 

5 

0:01:49 

2 

3:23:09 

86.4% 

1.0% 

0:02:21 

2 

8 

2 

0:00:54 

8 

3:24:57 

87.2% 

0.8% 

0:01:48 

8 

2 

1 

0:02:21 

5 

3:25:23 

87.4% 

0.2% 

0:00:26 

5 

5 

1 

0:00:26 

6 

3:25:23 

87.4% 

0.0% 

0:00:00 

6 

6 

0 

Table  21.  MYMC  PRE:  Details  by  Process. 
MYMC  POST  (N  =  1  observation  -  LPN): 


Nurse  Shadowing:  Study  Time  Period  =  POST,  Location  =  MYMC 


Processes  ranked  by  Total  Time  per  PID 

PID  ranked  by  Count 

PID 

cumulative 

time, 

hh:mm:ss 

cumulative 

%  of  time 

%  of 

Total  Time 
per  PID 

Total  Time 
/  PID, 

hh:m  m  :ss 

PID 

PID 

Count  of 
Activity 
IDs 

avg.  time  / 
AID, 

hh:mm:ss 

3 

1:42:04 

39.9% 

39.9% 

1:42:04 

3 

7 

44 

0:00:48 

1 

2:40:13 

62.6% 

22.7% 

0:58:09 

1 

4 

28 

0:00:59 

7 

3:15:08 

76.2% 

13.6% 

0:34:55 

7 

1 

22 

0:02:39 

4 

3:42:51 

87.1% 

10.8% 

0:27:43 

4 

3 

10 

0:10:12 

9 

3:58:57 

93.4% 

6.3% 

0:16:06 

9 

9 

4 

0:04:01 

2 

3:58:57 

93.4% 

0.0% 

0:00:00 

2 

2 

0 

5 

3:58:57 

93.4% 

0.0% 

0:00:00 

5 

5 

0 

6 

3:58:57 

93.4% 

0.0% 

0:00:00 

6 

6 

0 

8 

3:58:57 

93.4% 

0.0% 

0:00:00 

8 

8 

0 

10 

3:58:57 

93.4% 

0.0% 

0:00:00 

10 

10 

0 

Table  22.  MYMC  POST:  Details  by  Process. 
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Appendix  5  -  VLER  Patient  Letter 


—  Conemaugh 
Health  System 


Dear  Veteran, 


V  I 


There  Is  a  new  way  to  share  in  for  mail  on  with  your  healtbrare  providers  in  southwestern 
Hunrvsylvnnia! 


The  Virtual  lifetime  Electronic  Record  (VlER)  health  program  will  allow  you  to  Share  your 
electronic  health  information,  via  the  Nationwide  Health  Information  Network  (hi  wl  llhl),  with 
you i  providers  who  am  members  of  the  Conemaugh  Health  System  and  the  lames  F,  Van  Zsndt 
VA  Medical  Center  in  Altoona,  PA 


Wc  eivtle  you  to  sign  up  lur  the  VLER  llcalih  program  toy  completing  and  si^neis  the  enclosed 
forms  After  you  ^oin.  your  health  care  providers  will  be  able  10  see  your  current  medical  Issue  s 
emergency  room  reports,  discharge  summaries,  meditations,  allergies,  and  other  important 
infoimatlnn  in  your  medical  record  a:  necessary  for  your  treatment.  This  information  will  be 
used  to  Improve  th»  care  you  receive  from  your  providers,  and  avoid  duplicating  services. 
Access  to  your  information  will  be  sir  ctly  controlled  and  protected  using  data  security 
standards  commonly  used  in  thp  banting  industry. 

Once  you  have  comp, Vied  and  signed  the  forms  we  have  included,  please  (eLurn  them  in  the 
envelope  provided.  Signing  Lhese  forms  means  that  you  are  agreeing  to  shore  your  health 
Information  among  health  care  providers  in  the  NwtflN  Exchange  who  provide  your  medical 
treatment 


It  you  have  questions,  please  contact: 

John  Hargreaves,  Conemaugh  Health  system  ai  (8 id)  ?69*5277,  and/or 

Barbara  fiabiak,  VLLH  Coordinator,  Ja-nns  F  Van  Zandl  VA  Medical  Center,  at  (814)  943  SIC-' 

Ext.  8021 


Wc  a>e  proud  to  serve  veterans  that  :o/  v*d  our  country! 


Srott  Becker 

Chief  F«ecutrve  OITk.** 

Conemaugh  Health  System 


William  Mills 


U&ti 


(vied rear  Center  Director 

lames  EL  Van  Lh rdt  VA  Medical  Center 


s‘  ikki  Ati <te  nci  to  join.  iic  'K'thing.  vcv»  fm*Ui  mofd  eee  not  ik  resnre  •*st>>  o-'w-  ‘nutnco't fcaSi*-,  wa; 
I tvpmgnwe  ItiwiaycMllM^jkrllAcCmilWMMv/'aaygiirpM^  nw  fvtxtrt  CO: -  I lv  VS  c.* 
.•Oivv/i  br-fts  !ffoo\& onv  <n/pai  mrilMilvritatgc  rotveimn  /rc-i>v  contest  far  VA  A  -Hi»e  of 
Mto.v.< Office- crtfiia)  VAT  «.« 
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Appendix  6  -  Testing  Summary  Report 


M  ■  ■  ■  ■  ■  B® 
B  B  B  B  B  B  B 


CCHIT 


B 

B  B  B  B  B  B  B 
B  B  B  B  B  B  B 


Certification  Commission 
for  Health  Information 
Technology 


eHealth  Exchange  Participant 
Testing 

Test  Summary  Report  -  2010  & 
2011 

Conemaugh  Health  System 


10/25/2013 
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Document  Change  History 


Version  Nbr 

Date 

Change  Description 

Changed  By 

1.0 

10/25/2013 

Initial  Report  for  2010 

Heather  Geier,  CCH1T 

1.1 

10/25/2013 

Merged  Report  for  2010  &  201 1  (Initial  Report 
for  2011) 

Heather  Geier,  CCH1T 
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Summary  Report  Overview 


Date  of  completion 

September  3,  2013 

For 

Conemaugh  Health  System 

Applicant  Primary 
contact 

John  Hargreaves 

iharqrea(S)conemauqh.orq 

(814)  269-5277 

Gateway 
Implementation 
Technology  Vendor, 
if  applicable 

Northrop  Grumman  Information  Systems 

Gateway  Name  and 
Version 

CONNECT  3.3. 1.3 

Scope  of  testing 

Conemaugh  Health  System  tested  2010  and  2011  versions  for  eHealth  Exchange 
Participant  Testing. 

Testing  validated  adherence  to  the  following  eHealth  Exchange  specifications 
(unless  otherwise  noted): 

•  Patient  Discovery,  version  1 .0  (2010)  &  version  2.0  (2011) 

•  Query  for  Documents,  version  2.0  (2010)  &  version  3.0  (2011) 

•  Retrieve  Documents,  version  2.0  (2010)  &  version  3.0  (2011) 

•  Messaging  Platform,  version  2.0  (2010)  &  version  3.0  (2011) 

•  Authentication  Framework,  version  2.0  (2010)  &  version  3.0  (201 1) 

Summary  of  test 
results 

PASS:  Conemaugh  Health  System  successfully  completed  and  passed  eHealth 
Exchange  Participant  Testing  for  2010  &201 1,  with  the  exception  of  known  issues 
with  the  CONNECT  gateway  for  versions  lower  than  4.0. 
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Test  Results  Summary 


2010  Version 

Smoke  Test  Cases 


Service  Set 

Scenario 

Test  Case 

Status 

Analysis/Notes 

SS:  PRL-0000.0- 
2010 

TS:  PRL -1-0000.0- 
2010 

TC:  PD-I-0000. 0-2010 

PASS 

SS:  PRL-0000.0- 
2010 

TS:  PRL-I-0000.0- 
2010 

TC:  QD-I-0000.0-2010 

PASS 

SS:  PRL-0000.0- 
2010 

TS:  PRL-I-0000.0- 
2010 

TC:  RD-I-0000.0-2010 

PASS 

SS:  PRL-0000.0- 
2010 

TS:  PRL-R-0000.0- 
2010 

TC:  PD-R-0000.0-2010 

PASS 

SS:  PRL-0000.0- 
2010 

TS:  PRL-R-0000.0- 
2010 

TC:  QD-R-0000.0-20 1 0 

PASS 

SS:  PRL-0000.0- 
2010 

TS:  PRL-R-0000.0- 
2010 

TC:  RD-R-0000.0-2010 

PASS 

2011  Version 

Smoke  Test  Cases 


Service  Set 

Scenario 

Test  Case 

Status 

Analysis/Notes 

SS:  PRL-0000.0- 
2011 

TS:  PRL-1-0000.0- 
2011 

TC:  PD-I-0000. 0-201 1 

PASS* 

*  Manually  override  status  to  a  Pass 
with  a  reason  of  “Failing  only  because 
SSN  was  part  of  message.  SSN  was 
verified  to  be  sent  correctly” 

SS:  PRL-0000.0- 
2011 

TS:  PRL-I-0000.0- 
2011 

TC:  QD-I-0000.0-2011 

PASS 

SS:  PRL-0000.0- 
2011 

TS:  PRL-I-0000.0- 
2011 

TC:  RD-I-0000. 0-2011 

PASS 

SS:  PRL-0000.0- 
2011 

TS:  PRL-R-0000.0- 
2011 

TC:  PD-R-0000.0-201 1 

PASS 

SS:  PRL-0000.0- 
2011 

TS:  PRL-R-0000.0- 
2011 

TC:  QD-R-0000. 0-201 1 

PASS 

SS:  PRL-0000.0- 
2011 

TS:  PRL-R-0000.0- 
2011 

TC:  RD-R-0000. 0-2011 

PASS 

Security  Test  Cases 


Service  Set 

Scenario 

Test  Case 

Status 

Analysis/Notes 

SS:  PRL-0006.0- 
2011 

TS:  PRL-R-0006.0- 
2011 

TC:  MAQD-R-0003.000- 
2011 

PASS 

SS:  PRL-0006.0- 
2011 

TS:  PRL-R-0006.0- 
2011 

TC:  MAQD-R-0003 .101- 
2011 

PASS 

SS:  PRL-0006.0- 
2011 

TS:  PRL-R-0035.0- 
2011 

TC:  MAQD-R-0003. 301- 
2011 

PASS* 

*  Manually  override  status  to  a  Pass 
with  a  reason  of  “Known  Issue” 

SS:  PRL-0006.0- 
2011 

TS:  PRL-R-0036.0- 
2011 

TC:  MAQD-R-0003. 302- 
2011 

PASS 

SS:  PRL-0006.0- 
2011 

TS:  PRL-R-0036.0- 
2011 

TC:  MAQD-R-0003. 303- 
2011 

PASS 

SS:  PRL-0006.0- 
2011 

TS:  PRL-R-0037.0- 
2011 

TC:  MAQD-R-0003. 306- 
2011 

PASS 
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SS:  PRL-0006.0- 
2011 

TS:  PRL-R-0037.0- 
2011 

TC:  MAQD-R-0003.307- 
2011 

PASS 

SS:  PRL-0006.0- 
2011 

TS:  PRL-R-0038.0- 
2011 

TC:  MAQD-R-0003.308- 
2011 

PASS 

SS:  PRL-001 1.0- 
2011 

TS:  PRL-R-0040.0- 
2011 

TC:  MAQD-R-0003.315- 
2011 

PASS 

SS:  PRL-001 1.0- 
2011 

TS:  PRL-R-0040.0- 
2011 

TC:  MAQD-R-0003 .316- 
2011 

PASS 

SS:  PRL-001 1.0- 
2011 

TS:  PRL-R-0041.0- 
2011 

TC:  MAQD-R-0003. 317- 
2011 

PASS 

SS:  PRL-001 1.0- 
2011 

TS:  PRL-R-0043.0- 
2011 

TC:  MAQD-R-0003. 325- 
2011 

PASS 

SS:  PRL-001 1.0- 
2011 

TS:  PRL-R-0044.0- 
2011 

TC:  MAQD-R-0003. 326- 
2011 

PASS 

SS:  PRL-0012.0- 
2011 

TS:  PRL-R-0046.0- 
2011 

TC:  MAQD-R-0003. 401- 
2011 

PASS 

SS:  PRL-0013.0- 
2011 

TS:  PRL-R-0052.0- 
2011 

TC:  MAQD-R-0003 .42 1- 
2011 

PASS* 

*Manually  override  status  to  a  Pass 
with  a  reason  of  “Known  Issue” 

SS:  PRL-0013.0- 
2011 

TS:  PRL-R-0053.0- 
2011 

TC:  MAQD-R-0003. 423- 
2011 

PASS 

SS:  PRL-001 3.0- 
2011 

TS:  PRL-R-0053.0- 
2011 

TC:  MAQD-R-0003. 424- 
2011 

PASS* 

*Manually  override  status  to  a  Pass 
with  a  reason  of  “Known  Issue” 

SS:  PRL-001 3.0- 
2011 

TS:  PRL-R-0054.0- 
2011 

TC:  MAQD-R-0003. 426- 
2011 

PASS 

SS:  PRL-001 3.0- 
2011 

TS:  PRL-R-0054.0- 
2011 

TC:  MAQD-R-0003. 427- 
2011 

PASS 
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